-- Logs begin at Fri 2022-08-26 14:53:01 UTC, end at Fri 2022-08-26 16:13:54 UTC. -- Aug 26 15:30:03.673303 dv-cinder-831247-24-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 26 15:30:03.683488 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 26 15:30:03.684648 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 26 15:30:03 2022] *** Aug 26 15:30:03.684648 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 26 15:30:03.684648 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 26 15:30:03.684648 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: nodename: dv-cinder-831247-24-iscsi Aug 26 15:30:03.684648 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: machine: x86_64 Aug 26 15:30:03.684648 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: clock source: unix Aug 26 15:30:03.684648 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: pcre jit disabled Aug 26 15:30:03.684648 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: detected number of CPU cores: 4 Aug 26 15:30:03.684648 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: current working directory: / Aug 26 15:30:03.684648 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: detected binary path: /usr/bin/uwsgi-core Aug 26 15:30:03.685874 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: your processes number limit is 47680 Aug 26 15:30:03.685874 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: your memory page size is 4096 bytes Aug 26 15:30:03.685874 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: detected max file descriptor number: 2048 Aug 26 15:30:03.685874 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: lock engine: pthread robust mutexes Aug 26 15:30:03.685874 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: thunder lock: enabled Aug 26 15:30:03.685874 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 26 15:30:03.685874 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 26 15:30:03.724756 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: Python main interpreter initialized at 0x55a577d739c0 Aug 26 15:30:03.724756 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: python threads support enabled Aug 26 15:30:03.724756 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: your server socket listen backlog is limited to 100 connections Aug 26 15:30:03.724756 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: your mercy for graceful operations on workers is 90 seconds Aug 26 15:30:03.725098 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: mapped 403077 bytes (393 KB) for 2 cores Aug 26 15:30:03.725174 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: *** Operational MODE: preforking *** Aug 26 15:30:03.725450 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: *** uWSGI is running in multiple interpreter mode *** Aug 26 15:30:03.725450 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: spawned uWSGI master process (pid: 82213) Aug 26 15:30:03.725657 dv-cinder-831247-24-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 26 15:30:03.726186 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: spawned uWSGI worker 1 (pid: 82214, cores: 1) Aug 26 15:30:03.726521 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: spawned uWSGI worker 2 (pid: 82215, cores: 1) Aug 26 15:30:03.726695 dv-cinder-831247-24-iscsi devstack@keystone.service[82213]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 26 15:30:08.161396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 15:30:08.161396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: from cryptography.utils import int_from_bytes Aug 26 15:30:08.162233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 15:30:08.162233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: from cryptography.utils import int_from_bytes Aug 26 15:30:08.163238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 15:30:08.163238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: from cryptography.utils import int_from_bytes Aug 26 15:30:08.164033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 15:30:08.164033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: from cryptography.utils import int_from_bytes Aug 26 15:30:08.212761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 15:30:08.213021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 15:30:08.213137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] command line args: [] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 15:30:08.213447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 15:30:08.213583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ================================================================================ {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 15:30:08.213800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] admin_token = **** {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.213936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 15:30:08.214037 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] config_dir = [] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.214151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 15:30:08.214266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.214381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] command line args: [] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 15:30:08.214501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] config_source = [] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.214608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] debug = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.214911 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 15:30:08.216044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.218190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ================================================================================ {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 15:30:08.218190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] admin_token = **** {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.218190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] config_dir = [] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.218541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.218660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.218808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.218913 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] config_source = [] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.218990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] debug = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219182 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219249 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] list_limit = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] log_config_append = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219574 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219636 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] log_dir = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] log_file = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.219829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] list_limit = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.220341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.220448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] log_config_append = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.220512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.220579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.220639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.220695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] log_dir = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.220695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] log_file = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.220803 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.220803 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.220916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.220999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221090 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221904 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.221960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222357 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.98/identity {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222357 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] publish_errors = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.98/identity {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.222980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] publish_errors = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] standard_threads = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] use_journal = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] use_json = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] standard_threads = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] use_stderr = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.223945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] use_syslog = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.224068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.224126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.224183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.224244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.224308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.224308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] use_journal = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.224496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.224555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] use_json = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.224630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.224688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] use_stderr = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.224688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] use_syslog = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.224797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.224862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:30:08.224920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.224981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.225044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.229044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.229178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.229178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.230912 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.231450 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.231532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.231826 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.232172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.232263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.232343 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.232421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.232494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.232564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.232639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.232718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.232796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.232868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.232939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.233857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.233957 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234258 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.234981 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.235539 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.235636 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.235706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.235782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.235862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.235952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.236022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.236091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.236172 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.236253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.236337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.236409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.236488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.236596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.236894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.237046 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.237189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.237320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.237448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.237575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.237710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.237839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.237976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.238185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.238266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.238384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.238564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.238698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.238834 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.238920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.239042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.239165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.239361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.239483 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.239619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.239704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.239832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.240057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.240136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.240245 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.240332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.240444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.240531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.240643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.240757 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.240834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.240970 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.241108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.241192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.241306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.241306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.241542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.241666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.241755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.241819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.241879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.241938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.241996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.242759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.243006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.243218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.243389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.243471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.243639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.243737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.243856 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244774 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.244974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.245123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.245198 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.245309 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.245387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.245447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.245569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.245656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.245779 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.245861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.245977 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246135 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246845 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246917 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.246988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.247950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.248960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249187 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249729 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249803 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.249995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.250116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.250193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.250318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.250397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.250468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.250584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.250660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.250775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] auth.external = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.250860 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.250934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.251005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.251075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.251147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.251224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.251380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.251467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] auth.password = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.251549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.251623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] auth.token = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.251699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.251772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.251849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.252004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.252094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.252169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.252240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] auth.external = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.252312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.252428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.252505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253388 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] auth.password = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] credential.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] auth.token = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253906 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.253984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.254058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.254140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.254214 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.254286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.254366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.254447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.254567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.254643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.254715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.254831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] credential.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.254914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.255043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.255121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.255238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.255315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.255430 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.255508 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.255508 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.255765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.255936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.256016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.256132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.256209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.256337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] federation.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.256416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.256534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.256534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.256712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.256801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.256870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.256987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.257063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.257185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.257269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.257412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.257523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.257622 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.257781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.257882 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.257882 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258379 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] federation.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.258949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.259041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.259185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.259273 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.259433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.259517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.259666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.259759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.259926 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.260010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.260179 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.260261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.260431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.260527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.260664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.260745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.260895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.260976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.261127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.261216 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.261372 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.261464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.261617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.261696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.261858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.261947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.262111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.262202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.262346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.262422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.262591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.262679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.262840 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.262919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.263077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.263160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.263306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.263398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.263544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.263966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.264091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.264169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.264286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.264379 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.264548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.264630 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.264748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.264832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.264900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.265033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.265113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.265230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.265307 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.265427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.265514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.265583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.265700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.265777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.265887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.266016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.266136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.266258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.266339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.266495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.266578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.266737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.266892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.266975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.267138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.267218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.267380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.267460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.267630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.267715 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.267894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.267980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.268132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.268823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.268962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.269066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.269205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.269285 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.269440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.269523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.269674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.269758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.269925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.270005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.270159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.270249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.270402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.270482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.270638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.270717 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.270876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.270955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.271107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.271188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.271347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.271467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.271603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.271683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.271834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.272005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.272108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.272188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.272342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.272423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.272591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.272675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.272827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.272909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.273057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.273140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.273335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.273416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.273584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.273665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.273818 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.273965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.274055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.274348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.274481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.274562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.274635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.274707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.274851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.274953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.275034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.275210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.275290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.275440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.275554 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.277023 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.277130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.277130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.277268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.277341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.277430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.277508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.277580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.277580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.278005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.278005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.278851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.278851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.278989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279285 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279408 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279468 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279726 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.279986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.280046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.280106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.280278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.280360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.280433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.280504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.280576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.280648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.280728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.280802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] resource.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.280923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.281967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.282039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.282122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.282196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.282269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.282352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.282480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] role.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.282610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.282755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] role.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.282889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.283021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.283149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] resource.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.283279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.283458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.283605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.283722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.283831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.283916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.283979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.284039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.284182 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.284262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.284335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.284407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.284479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.284633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.284729 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.284903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.285095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.285175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] role.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.285384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] role.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.285463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.285622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.285703 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.285877 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.285958 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.286133 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.286213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.286389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.286469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.286642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.286832 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.286911 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.287071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.287159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.287323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.287403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.287577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.287657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.287829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.288002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.288105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.288176 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.288342 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.288412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.288598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.288667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.288848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.288916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.289089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.289281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.289358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.289524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.289593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.289759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.289827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.290011 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.290108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.290264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.290343 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.290510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.290588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.290745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.290833 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.290981 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.291061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.291214 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.291293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.291450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] token.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.291528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.291687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.291760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.291948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.292026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.292185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.292269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.292441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.292509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.292690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.292770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.293263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.293263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.293407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.293480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.293592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.293673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.293838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.293926 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.294078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.294158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.294322 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] token.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.294546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.294626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.294808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.295000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.295080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.295565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.295740 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.295820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.296079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.296161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.296330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.296554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.296636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.296820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.297024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.297112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.297293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.297490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.connection = **** {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.297570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.297771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.297921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.298055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.298293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.298381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.298541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.298656 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.298816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.298936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.299083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.connection = **** {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.299280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.299404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.299552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.299664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.299746 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.299929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.300116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.300198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.300410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.300553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.300655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.300896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.300984 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.301139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.301389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.301468 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.301662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.301786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.301877 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.302086 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.302167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.302348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.302490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.302590 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.302773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.302854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.303043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.303123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.303292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.303371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.303532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.303613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:30:08.303708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 15:30:08.303789 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 15:30:08.335196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.335474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.335601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.335712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.335843 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.336038 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.344432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.344565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.379956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.380177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.380304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.380721 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.443383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.449231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.497015 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.497243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.501753 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.501994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.656501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.656739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.656959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.657188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.657401 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.657617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.657805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.658047 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.659576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.659576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.659576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.659576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.690261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.691041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.691149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.691232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.691307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.691417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.691501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.691637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.691786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.691926 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.692010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.692163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.692260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.692413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.703732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.703896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.716806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.717044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.723653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.723795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.723904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.723979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.724052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.724141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.724218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.724291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.724364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.724436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.724517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.724517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:30:08.737824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.738035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 15:30:08.738383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.738603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 15:30:08.738937 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.739205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.739448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.739536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.739695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 15:30:08.740005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.740090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.740272 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 15:30:08.740557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.740832 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.741105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.741647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.749613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.749914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.750192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.750486 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.754942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.755446 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.755811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.756144 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.758505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.779154 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.779154 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.779154 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.779154 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.779154 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.779154 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.779154 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.779660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.779740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.779740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.779740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.779740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.780022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.780022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.780022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.780022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.780022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.780022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.780022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.780424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.780424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.780424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.780424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.787133 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.788683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.788975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.791972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.793469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.793663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.793763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.793949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.794235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.798716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.799007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.799301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.801412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.801728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.805790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.806079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.806281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.806479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.806620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.806751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.806894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.807163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.807570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.807750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.810695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.814688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.814688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.814688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.814688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.814688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.814688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.815754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.825827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.826192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.830082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.830837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.831160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.833326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.833626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.835102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.837848 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.838375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.838685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.840982 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.842963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.844268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.844575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.844866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.845155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.845447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.845751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.846201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.850049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.850363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.850667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.850965 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.851257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.851559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.862833 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.863133 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.863508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.863711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.868163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.868475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.868766 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.869042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.870654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.872209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.872485 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.875134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.883400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.883400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.883400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.883793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.883985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.884122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.884122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.884122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.884122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.884122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.884122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.884786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.887537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.887825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.888139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.888421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.888705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.888987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.889263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.893304 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.893602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.893898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.894191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.894479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.894763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.895049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.904688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.907675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.911081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.911252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.912538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.913990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.914140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.914285 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.914541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.917068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.918524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.920048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.920583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.920885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.921244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.922695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.926218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.926504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.926796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.927078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.927993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.929466 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.932409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.932704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.933003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.933291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.934951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.935242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.935531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.941340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.941651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.941943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.942409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.942698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.942975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.943252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.943541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.943818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.944116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.944397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.944670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.944960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.945230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.945494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.945760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.946019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.948855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:30:08.949148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.949439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.949727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.950010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.950303 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.950586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.951501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.951501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.951501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.952140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.952140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.952275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.952519 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:30:08.983788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a577d739c0 pid: 82214 (default app) Aug 26 15:30:08.987783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a577d739c0 pid: 82215 (default app) Aug 26 15:30:08.987990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873577d3-4a3d-48ec-a4b0-413b22af81ad None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:08.988287 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873577d3-4a3d-48ec-a4b0-413b22af81ad None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:08.988585 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873577d3-4a3d-48ec-a4b0-413b22af81ad None None] PATH_INFO: `/v3` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:08.992728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1/1] 10.222.0.98 () {50 vars in 791 bytes} [Fri Aug 26 15:30:08 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:16.757792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b385ea7-6e3a-4979-9a03-0969bb7b1d47 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:16.758338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b385ea7-6e3a-4979-9a03-0969bb7b1d47 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:16.758338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b385ea7-6e3a-4979-9a03-0969bb7b1d47 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:16.762563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1/2] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:16.768433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66828f6e-6f45-475b-882e-7266248c710e None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:16.769385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66828f6e-6f45-475b-882e-7266248c710e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:16.769551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66828f6e-6f45-475b-882e-7266248c710e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:16.840594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: INFO dbcounter [None req-66828f6e-6f45-475b-882e-7266248c710e None None] Registered counter for database keystone Aug 26 15:30:16.862658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG oslo_db.sqlalchemy.engines [None req-66828f6e-6f45-475b-882e-7266248c710e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82214) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 15:30:16.869091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writer thread running {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 15:30:17.068157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG passlib.handlers.bcrypt [None req-66828f6e-6f45-475b-882e-7266248c710e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=82214) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 15:30:17.068585 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG passlib.handlers.bcrypt [None req-66828f6e-6f45-475b-882e-7266248c710e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82214) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 15:30:17.129690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-66828f6e-6f45-475b-882e-7266248c710e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:17.129690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-66828f6e-6f45-475b-882e-7266248c710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:17.260975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 2/3] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 26 15:30:17.724150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b385ea7-6e3a-4979-9a03-0969bb7b1d47 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:17.793440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: INFO dbcounter [None req-6b385ea7-6e3a-4979-9a03-0969bb7b1d47 None None] Registered counter for database keystone Aug 26 15:30:17.813978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG oslo_db.sqlalchemy.engines [None req-6b385ea7-6e3a-4979-9a03-0969bb7b1d47 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82215) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 15:30:17.821045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writer thread running {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 15:30:17.941281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b385ea7-6e3a-4979-9a03-0969bb7b1d47 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:17.945688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae1790-378a-435a-8d7e-c02a1596fbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:17.946323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae1790-378a-435a-8d7e-c02a1596fbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:17.946610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae1790-378a-435a-8d7e-c02a1596fbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:17.946886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae1790-378a-435a-8d7e-c02a1596fbc7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:17.955770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae1790-378a-435a-8d7e-c02a1596fbc7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:18.007391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5fae1790-378a-435a-8d7e-c02a1596fbc7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 15:30:18.007741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae1790-378a-435a-8d7e-c02a1596fbc7 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:18.018312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 2/4] 10.222.0.98 () {58 vars in 1324 bytes} [Fri Aug 26 15:30:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:30:18.024570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66828f6e-6f45-475b-882e-7266248c710e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:18.041202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66828f6e-6f45-475b-882e-7266248c710e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:18.046087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b976f-5232-401b-9b88-e0236c9fbdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:18.047814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b976f-5232-401b-9b88-e0236c9fbdb2 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:18.048162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b976f-5232-401b-9b88-e0236c9fbdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:18.048442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b976f-5232-401b-9b88-e0236c9fbdb2 admin admin] PATH_INFO: `/v3/projects/86d26d8b55fb4a2b877ae676d9b1d379` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:18.050089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b976f-5232-401b-9b88-e0236c9fbdb2 admin admin] RBAC: Authorizing `identity:get_project(project_id=86d26d8b55fb4a2b877ae676d9b1d379)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:18.103433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bc5b976f-5232-401b-9b88-e0236c9fbdb2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 15:30:18.103816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b976f-5232-401b-9b88-e0236c9fbdb2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:18.108555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 3/5] 10.222.0.98 () {56 vars in 1207 bytes} [Fri Aug 26 15:30:18 2022] GET /identity/v3/projects/86d26d8b55fb4a2b877ae676d9b1d379 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:20.025333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b523d-5dfe-4e59-adc2-a1528a4e21c5 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:20.025845 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b523d-5dfe-4e59-adc2-a1528a4e21c5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:20.025931 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b523d-5dfe-4e59-adc2-a1528a4e21c5 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:20.029973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 3/6] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:30:20.036020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a863a9-29d1-4e3e-9c71-9505713a92fb None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:20.036426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a863a9-29d1-4e3e-9c71-9505713a92fb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:20.036576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a863a9-29d1-4e3e-9c71-9505713a92fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:20.056702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f6a863a9-29d1-4e3e-9c71-9505713a92fb None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:20.058254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f6a863a9-29d1-4e3e-9c71-9505713a92fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:20.066467 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 4/7] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 26 15:30:20.339805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b523d-5dfe-4e59-adc2-a1528a4e21c5 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:20.352274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b523d-5dfe-4e59-adc2-a1528a4e21c5 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:20.356614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfec5a-45f7-4aef-8867-d5a349acbbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:20.357235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfec5a-45f7-4aef-8867-d5a349acbbec admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:20.357547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfec5a-45f7-4aef-8867-d5a349acbbec admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:20.357825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfec5a-45f7-4aef-8867-d5a349acbbec admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:20.366270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfec5a-45f7-4aef-8867-d5a349acbbec admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:20.368132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfec5a-45f7-4aef-8867-d5a349acbbec admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:20.377793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 4/8] 10.222.0.98 () {58 vars in 1324 bytes} [Fri Aug 26 15:30:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:30:20.384233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a863a9-29d1-4e3e-9c71-9505713a92fb None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:20.393599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a863a9-29d1-4e3e-9c71-9505713a92fb None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:20.398665 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21344a-8033-4d49-81e8-c36ca066a580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:20.399324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21344a-8033-4d49-81e8-c36ca066a580 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:20.399609 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21344a-8033-4d49-81e8-c36ca066a580 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:20.399948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21344a-8033-4d49-81e8-c36ca066a580 admin admin] PATH_INFO: `/v3/users/3fea9e7c5b754acaa0233c15976ac3b3` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:20.401312 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21344a-8033-4d49-81e8-c36ca066a580 admin admin] RBAC: Authorizing `identity:get_user(user_id=3fea9e7c5b754acaa0233c15976ac3b3)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:20.403163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21344a-8033-4d49-81e8-c36ca066a580 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:20.404769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 5/9] 10.222.0.98 () {56 vars in 1198 bytes} [Fri Aug 26 15:30:20 2022] GET /identity/v3/users/3fea9e7c5b754acaa0233c15976ac3b3 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:22.383003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefacf91-144b-4654-8738-6749310b4374 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:22.383523 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefacf91-144b-4654-8738-6749310b4374 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:22.383605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefacf91-144b-4654-8738-6749310b4374 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:22.384937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 5/10] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:30:22.390492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1cac-1ac4-4839-8189-c0025bbc91af None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:22.390982 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1cac-1ac4-4839-8189-c0025bbc91af None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:22.391067 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1cac-1ac4-4839-8189-c0025bbc91af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:22.409965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9bcd1cac-1ac4-4839-8189-c0025bbc91af None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:22.411178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9bcd1cac-1ac4-4839-8189-c0025bbc91af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:22.438283 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600604d-cebe-4874-ba14-59825463dc77 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:22.438716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600604d-cebe-4874-ba14-59825463dc77 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:22.439000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600604d-cebe-4874-ba14-59825463dc77 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:22.440502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 6/11] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:30:22.446174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35e831-d2a5-40c7-a2e1-fd96a9199b83 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:22.446459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35e831-d2a5-40c7-a2e1-fd96a9199b83 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:22.446737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35e831-d2a5-40c7-a2e1-fd96a9199b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:22.488811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG passlib.handlers.bcrypt [None req-ae35e831-d2a5-40c7-a2e1-fd96a9199b83 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=82215) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 15:30:22.489218 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG passlib.handlers.bcrypt [None req-ae35e831-d2a5-40c7-a2e1-fd96a9199b83 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82215) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 15:30:22.502283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 6/12] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:22.535233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ae35e831-d2a5-40c7-a2e1-fd96a9199b83 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:22.537055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ae35e831-d2a5-40c7-a2e1-fd96a9199b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:22.641050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 7/13] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:30:22.778742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd1cac-1ac4-4839-8189-c0025bbc91af None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:22.803707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:22.907356 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd1cac-1ac4-4839-8189-c0025bbc91af None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:22.912325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35e831-d2a5-40c7-a2e1-fd96a9199b83 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:22.945301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9483c-b654-403e-93b5-75784d6594fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:22.945940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9483c-b654-403e-93b5-75784d6594fb None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:22.946222 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9483c-b654-403e-93b5-75784d6594fb None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:22.946498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9483c-b654-403e-93b5-75784d6594fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:22.959252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9483c-b654-403e-93b5-75784d6594fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-a9c9483c-b654-403e-93b5-75784d6594fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:22.968103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:22.969213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:22.970155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 7/14] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:30:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:22.975077 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9483c-b654-403e-93b5-75784d6594fb None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:23.024422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35e831-d2a5-40c7-a2e1-fd96a9199b83 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:23.027788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:23.063068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:23.065826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfbba0-e4be-4935-90aa-7f7b78a3087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:23.067262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfbba0-e4be-4935-90aa-7f7b78a3087e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:23.067550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfbba0-e4be-4935-90aa-7f7b78a3087e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:23.067828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfbba0-e4be-4935-90aa-7f7b78a3087e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:23.081234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9483c-b654-403e-93b5-75784d6594fb None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:23.119142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f19b4-1385-489f-9a6b-d76a9e694963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:23.119727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f19b4-1385-489f-9a6b-d76a9e694963 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:23.120062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f19b4-1385-489f-9a6b-d76a9e694963 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:23.120338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f19b4-1385-489f-9a6b-d76a9e694963 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:23.121017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f19b4-1385-489f-9a6b-d76a9e694963 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:23.121320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f19b4-1385-489f-9a6b-d76a9e694963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:23.122787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f19b4-1385-489f-9a6b-d76a9e694963 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:23.135805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 8/15] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:23.139384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfbba0-e4be-4935-90aa-7f7b78a3087e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:23.141469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfbba0-e4be-4935-90aa-7f7b78a3087e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:23.141580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f19b4-1385-489f-9a6b-d76a9e694963 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:23.244677 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 8/16] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 15:30:23.250536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfbba0-e4be-4935-90aa-7f7b78a3087e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:23.255780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f19b4-1385-489f-9a6b-d76a9e694963 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:23.257390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:23.292921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f331758-52cf-4b7e-908e-04ddab38334f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:23.293565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331758-52cf-4b7e-908e-04ddab38334f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:23.293849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331758-52cf-4b7e-908e-04ddab38334f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:23.294127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331758-52cf-4b7e-908e-04ddab38334f None admin] PATH_INFO: `/v3/users/3fea9e7c5b754acaa0233c15976ac3b3` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:23.295318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f331758-52cf-4b7e-908e-04ddab38334f None admin] RBAC: Authorizing `identity:get_user(user_id=3fea9e7c5b754acaa0233c15976ac3b3)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:23.296822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f331758-52cf-4b7e-908e-04ddab38334f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:23.298668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 9/17] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:23 2022] GET /identity/v3/users/3fea9e7c5b754acaa0233c15976ac3b3 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:23.304976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f331758-52cf-4b7e-908e-04ddab38334f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:23.321431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:23.366996 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfbba0-e4be-4935-90aa-7f7b78a3087e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:23.404389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff25f51-83dd-4df5-9e2c-501edfc715d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:23.405247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff25f51-83dd-4df5-9e2c-501edfc715d2 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:23.405537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff25f51-83dd-4df5-9e2c-501edfc715d2 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:23.405808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff25f51-83dd-4df5-9e2c-501edfc715d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:23.417800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f331758-52cf-4b7e-908e-04ddab38334f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:23.425197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff25f51-83dd-4df5-9e2c-501edfc715d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:23.427223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff25f51-83dd-4df5-9e2c-501edfc715d2 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-6ff25f51-83dd-4df5-9e2c-501edfc715d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:30:23.447588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:23.448748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:30:23.449674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:30:23.449674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:30:23.449674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:30:23.449674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 9/18] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:23.454517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff25f51-83dd-4df5-9e2c-501edfc715d2 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:23.458130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5841df-8f15-4f3a-bb2e-b6c2feb1fdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:23.458775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5841df-8f15-4f3a-bb2e-b6c2feb1fdc1 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:23.459060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5841df-8f15-4f3a-bb2e-b6c2feb1fdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:23.459350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5841df-8f15-4f3a-bb2e-b6c2feb1fdc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:23.460509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5841df-8f15-4f3a-bb2e-b6c2feb1fdc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:23.461986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5841df-8f15-4f3a-bb2e-b6c2feb1fdc1 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:23.463582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 10/19] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:23.469787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5841df-8f15-4f3a-bb2e-b6c2feb1fdc1 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:23.521290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:23.553528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:23.572513 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff25f51-83dd-4df5-9e2c-501edfc715d2 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:23.573581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5841df-8f15-4f3a-bb2e-b6c2feb1fdc1 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:23.609625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a11aaf-54ec-4074-bf2c-6b56a69e070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:23.610058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d10863-5b26-4241-bec2-573a5e711d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:23.610246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a11aaf-54ec-4074-bf2c-6b56a69e070a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:23.610533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a11aaf-54ec-4074-bf2c-6b56a69e070a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:23.610674 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d10863-5b26-4241-bec2-573a5e711d18 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:23.610828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a11aaf-54ec-4074-bf2c-6b56a69e070a None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:23.610967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d10863-5b26-4241-bec2-573a5e711d18 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:23.611229 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d10863-5b26-4241-bec2-573a5e711d18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:23.611458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a11aaf-54ec-4074-bf2c-6b56a69e070a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:23.611765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a11aaf-54ec-4074-bf2c-6b56a69e070a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:23.611996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d10863-5b26-4241-bec2-573a5e711d18 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.domain.id=default, user.id=3fea9e7c5b754acaa0233c15976ac3b3) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:23.612308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d10863-5b26-4241-bec2-573a5e711d18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:23.613210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a11aaf-54ec-4074-bf2c-6b56a69e070a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:23.613896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d10863-5b26-4241-bec2-573a5e711d18 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:23.620900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 11/20] 10.222.0.98 () {56 vars in 1322 bytes} [Fri Aug 26 15:30:23 2022] GET /identity/v3/role_assignments?user.id=3fea9e7c5b754acaa0233c15976ac3b3&scope.domain.id=default&role.id=6ba5a5add2174983be37258aad1ee404 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:23.633063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 10/21] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:25.919750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b10931-ada4-46b0-a066-5a381321a51c None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:25.920255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b10931-ada4-46b0-a066-5a381321a51c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:25.920255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b10931-ada4-46b0-a066-5a381321a51c None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:25.921589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 12/22] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:25.927376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b238f0-32d1-474a-a170-c7bdcb2c27bb None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:25.927785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b238f0-32d1-474a-a170-c7bdcb2c27bb None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:25.927974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b238f0-32d1-474a-a170-c7bdcb2c27bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:25.949962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-27b238f0-32d1-474a-a170-c7bdcb2c27bb None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:25.951186 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-27b238f0-32d1-474a-a170-c7bdcb2c27bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:26.086663 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 11/23] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:26.143732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3cffa-8453-4004-beda-e26a054ed207 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:26.145181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3cffa-8453-4004-beda-e26a054ed207 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:26.145872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3cffa-8453-4004-beda-e26a054ed207 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:26.148668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 13/24] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:26.151463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e13b77-130c-492e-a0a5-31b30e8586c9 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:26.152315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e13b77-130c-492e-a0a5-31b30e8586c9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:26.153092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e13b77-130c-492e-a0a5-31b30e8586c9 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:26.160374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 12/25] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:26.162704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6d699-9969-4fb6-a6d9-04784db8d840 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:26.163235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6d699-9969-4fb6-a6d9-04784db8d840 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:26.163732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6d699-9969-4fb6-a6d9-04784db8d840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:26.171063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca6f69-e9fc-496c-ad61-bc8e5d9fb145 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:26.173471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca6f69-e9fc-496c-ad61-bc8e5d9fb145 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:26.174851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca6f69-e9fc-496c-ad61-bc8e5d9fb145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:26.201050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-2be6d699-9969-4fb6-a6d9-04784db8d840 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:26.203240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2be6d699-9969-4fb6-a6d9-04784db8d840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:26.217865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-d5ca6f69-e9fc-496c-ad61-bc8e5d9fb145 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:26.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d5ca6f69-e9fc-496c-ad61-bc8e5d9fb145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:26.344405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:26.372586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 14/26] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:26.406556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 13/27] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:26.447212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c028aeb-0261-4e42-97ac-2dc760376630 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:26.447977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c028aeb-0261-4e42-97ac-2dc760376630 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:26.448491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c028aeb-0261-4e42-97ac-2dc760376630 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:26.450310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 15/28] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:26.460043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dba88-3fb0-4171-9533-aa97bb47b9de None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:26.460597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dba88-3fb0-4171-9533-aa97bb47b9de None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:26.461092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dba88-3fb0-4171-9533-aa97bb47b9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:26.483215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-a40dba88-3fb0-4171-9533-aa97bb47b9de None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:26.484788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a40dba88-3fb0-4171-9533-aa97bb47b9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:26.591087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 14/29] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:26.695186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c028aeb-0261-4e42-97ac-2dc760376630 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:26.770916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40dba88-3fb0-4171-9533-aa97bb47b9de None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:26.930095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:26.986134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c028aeb-0261-4e42-97ac-2dc760376630 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:27.048059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40dba88-3fb0-4171-9533-aa97bb47b9de None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:27.061039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:27.072002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a0bc0-2f2d-4329-b3eb-de53b53ff76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:27.072597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a0bc0-2f2d-4329-b3eb-de53b53ff76d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:27.084096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a0bc0-2f2d-4329-b3eb-de53b53ff76d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:27.084096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a0bc0-2f2d-4329-b3eb-de53b53ff76d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:27.094891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a0bc0-2f2d-4329-b3eb-de53b53ff76d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-748a0bc0-2f2d-4329-b3eb-de53b53ff76d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:27.104465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:27.105355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:27.106119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:27.107668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 16/30] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:30:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:27.108698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a0bc0-2f2d-4329-b3eb-de53b53ff76d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:27.110220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf86fc-7e1b-4d7e-80da-bc2bbcaec8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:27.110909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf86fc-7e1b-4d7e-80da-bc2bbcaec8a5 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:27.111187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf86fc-7e1b-4d7e-80da-bc2bbcaec8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:27.111467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf86fc-7e1b-4d7e-80da-bc2bbcaec8a5 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:27.119941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf86fc-7e1b-4d7e-80da-bc2bbcaec8a5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:27.119941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf86fc-7e1b-4d7e-80da-bc2bbcaec8a5 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:27.271203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a0bc0-2f2d-4329-b3eb-de53b53ff76d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:27.338816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d540af6-d376-4b86-9299-34f23e8e154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:27.339898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540af6-d376-4b86-9299-34f23e8e154c None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:27.339898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540af6-d376-4b86-9299-34f23e8e154c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:27.339898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540af6-d376-4b86-9299-34f23e8e154c None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:27.343892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d540af6-d376-4b86-9299-34f23e8e154c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:27.343892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d540af6-d376-4b86-9299-34f23e8e154c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:27.347415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:27.397802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 15/31] 10.222.0.98 () {60 vars in 1131 bytes} [Fri Aug 26 15:30:26 2022] POST /identity/v3/roles => generated 219 bytes in 628 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:30:27.398995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf86fc-7e1b-4d7e-80da-bc2bbcaec8a5 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:27.638913 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 17/32] 10.222.0.98 () {60 vars in 1131 bytes} [Fri Aug 26 15:30:27 2022] POST /identity/v3/roles => generated 225 bytes in 530 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:30:27.640406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d540af6-d376-4b86-9299-34f23e8e154c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:27.652877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf86fc-7e1b-4d7e-80da-bc2bbcaec8a5 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:27.706652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262d7fb-2ce6-4d20-9de9-1066e527ea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:27.709121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262d7fb-2ce6-4d20-9de9-1066e527ea6f None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:27.709121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262d7fb-2ce6-4d20-9de9-1066e527ea6f None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:27.709121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262d7fb-2ce6-4d20-9de9-1066e527ea6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:27.740037 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262d7fb-2ce6-4d20-9de9-1066e527ea6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:27.743637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262d7fb-2ce6-4d20-9de9-1066e527ea6f None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-f262d7fb-2ce6-4d20-9de9-1066e527ea6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:30:27.773829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:27.774783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:30:27.775606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:30:27.775606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:30:27.775606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:30:27.777585 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 16/33] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:27.779314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b493-abfc-4dd8-abb8-181e496ff148 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:27.779861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b493-abfc-4dd8-abb8-181e496ff148 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:27.780420 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b493-abfc-4dd8-abb8-181e496ff148 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:27.782313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 17/34] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:27.789384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2607800-4603-492f-831f-acd16f3adc31 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:27.790000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2607800-4603-492f-831f-acd16f3adc31 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:27.790479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2607800-4603-492f-831f-acd16f3adc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:27.841346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-a2607800-4603-492f-831f-acd16f3adc31 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:27.841346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a2607800-4603-492f-831f-acd16f3adc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:27.846081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d540af6-d376-4b86-9299-34f23e8e154c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:27.858063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:27.906682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d610578-ec1f-4a08-9bde-72d848929cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:27.906682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d610578-ec1f-4a08-9bde-72d848929cfa None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:27.906682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d610578-ec1f-4a08-9bde-72d848929cfa None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:27.906682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d610578-ec1f-4a08-9bde-72d848929cfa None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:27.907337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d610578-ec1f-4a08-9bde-72d848929cfa None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:27.907642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d610578-ec1f-4a08-9bde-72d848929cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:27.909661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d610578-ec1f-4a08-9bde-72d848929cfa None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:27.921710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 18/35] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:27.922832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d610578-ec1f-4a08-9bde-72d848929cfa None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:27.953704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 18/36] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:27.953704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2607800-4603-492f-831f-acd16f3adc31 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:28.000648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:28.055847 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d610578-ec1f-4a08-9bde-72d848929cfa None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:28.109706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0520e-a342-45f6-8bf4-f7c3d4c53cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:28.110253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0520e-a342-45f6-8bf4-f7c3d4c53cb0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:28.110535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0520e-a342-45f6-8bf4-f7c3d4c53cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:28.110817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0520e-a342-45f6-8bf4-f7c3d4c53cb0 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:28.111477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0520e-a342-45f6-8bf4-f7c3d4c53cb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:28.111795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0520e-a342-45f6-8bf4-f7c3d4c53cb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:28.113238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0520e-a342-45f6-8bf4-f7c3d4c53cb0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:28.118329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2607800-4603-492f-831f-acd16f3adc31 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:28.134765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 19/37] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:28.136475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795abe9-17e5-4f65-b18f-716bce2f1cbb None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:28.136772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795abe9-17e5-4f65-b18f-716bce2f1cbb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:28.137042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795abe9-17e5-4f65-b18f-716bce2f1cbb None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:28.138300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 20/38] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:28.140635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795abe9-17e5-4f65-b18f-716bce2f1cbb None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:28.169785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f1bb1-b1b1-4a51-96a4-3fa31c071bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:28.170612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f1bb1-b1b1-4a51-96a4-3fa31c071bc4 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:28.170999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f1bb1-b1b1-4a51-96a4-3fa31c071bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:28.171387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f1bb1-b1b1-4a51-96a4-3fa31c071bc4 None admin] PATH_INFO: `/v3/users/3fea9e7c5b754acaa0233c15976ac3b3` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:28.173412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f1bb1-b1b1-4a51-96a4-3fa31c071bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=3fea9e7c5b754acaa0233c15976ac3b3)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:28.175053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f1bb1-b1b1-4a51-96a4-3fa31c071bc4 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:28.178037 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 19/39] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:27 2022] GET /identity/v3/users/3fea9e7c5b754acaa0233c15976ac3b3 => generated 247 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:28.184336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1fbd2-6efa-4810-b67f-bdd53bb323e9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:28.184630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1fbd2-6efa-4810-b67f-bdd53bb323e9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:28.184920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1fbd2-6efa-4810-b67f-bdd53bb323e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:28.205581 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7bc1fbd2-6efa-4810-b67f-bdd53bb323e9 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:28.206840 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7bc1fbd2-6efa-4810-b67f-bdd53bb323e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:28.226051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:28.288780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 20/40] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:28.288780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1fbd2-6efa-4810-b67f-bdd53bb323e9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:28.392360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795abe9-17e5-4f65-b18f-716bce2f1cbb None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:28.414569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1fbd2-6efa-4810-b67f-bdd53bb323e9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:28.416526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:28.439101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2873c-4ed4-4063-9ca6-922649759faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:28.439668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2873c-4ed4-4063-9ca6-922649759faa None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:28.439987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2873c-4ed4-4063-9ca6-922649759faa None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:28.440353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2873c-4ed4-4063-9ca6-922649759faa None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:28.440929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2873c-4ed4-4063-9ca6-922649759faa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:28.442639 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2873c-4ed4-4063-9ca6-922649759faa None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:28.455557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0800e89-de59-48f3-8313-3d39835c158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:28.456261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0800e89-de59-48f3-8313-3d39835c158a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:28.456551 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0800e89-de59-48f3-8313-3d39835c158a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:28.456834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0800e89-de59-48f3-8313-3d39835c158a None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:28.458050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0800e89-de59-48f3-8313-3d39835c158a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:28.460003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0800e89-de59-48f3-8313-3d39835c158a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:28.466497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 21/41] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:28.467547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0800e89-de59-48f3-8313-3d39835c158a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:28.489776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:28.582561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 21/42] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:30:28 2022] POST /identity/v3/projects => generated 301 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:30:28.584427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2873c-4ed4-4063-9ca6-922649759faa None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:28.615555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0800e89-de59-48f3-8313-3d39835c158a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:28.684149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0eb50-07f4-4039-bcd6-403209b0e93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:28.685794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0eb50-07f4-4039-bcd6-403209b0e93d None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:28.686486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0eb50-07f4-4039-bcd6-403209b0e93d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:28.686973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0eb50-07f4-4039-bcd6-403209b0e93d None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:28.687764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0eb50-07f4-4039-bcd6-403209b0e93d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:28.689647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0eb50-07f4-4039-bcd6-403209b0e93d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:28.750019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2873c-4ed4-4063-9ca6-922649759faa None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:28.794494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 22/43] 10.222.0.98 () {60 vars in 1131 bytes} [Fri Aug 26 15:30:28 2022] POST /identity/v3/roles => generated 223 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:30:28.796364 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0eb50-07f4-4039-bcd6-403209b0e93d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:28.811961 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f6076-4102-41aa-bb69-36147a458d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:28.813533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f6076-4102-41aa-bb69-36147a458d5b None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:28.813644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f6076-4102-41aa-bb69-36147a458d5b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:28.816838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f6076-4102-41aa-bb69-36147a458d5b None admin] PATH_INFO: `/v3/domains/default/users/3fea9e7c5b754acaa0233c15976ac3b3/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:28.823281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f6076-4102-41aa-bb69-36147a458d5b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3fea9e7c5b754acaa0233c15976ac3b3, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:28.826320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f6076-4102-41aa-bb69-36147a458d5b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:28.913757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:28.925330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 22/44] 10.222.0.98 () {58 vars in 1361 bytes} [Fri Aug 26 15:30:28 2022] PUT /identity/v3/domains/default/users/3fea9e7c5b754acaa0233c15976ac3b3/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:30:28.958192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0eb50-07f4-4039-bcd6-403209b0e93d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:29.018341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2101d3-1ea4-4ad1-8f49-bd35f407aaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:29.018341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2101d3-1ea4-4ad1-8f49-bd35f407aaa3 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:29.018341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2101d3-1ea4-4ad1-8f49-bd35f407aaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:29.018341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2101d3-1ea4-4ad1-8f49-bd35f407aaa3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:29.021272 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2101d3-1ea4-4ad1-8f49-bd35f407aaa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:29.022808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2101d3-1ea4-4ad1-8f49-bd35f407aaa3 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:29.024958 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 23/45] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:29.031188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f6076-4102-41aa-bb69-36147a458d5b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:29.083533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:29.163287 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f6076-4102-41aa-bb69-36147a458d5b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:29.202517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b3f1-95fb-4de3-96cb-64a7655ab538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:29.203178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b3f1-95fb-4de3-96cb-64a7655ab538 None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:29.203472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b3f1-95fb-4de3-96cb-64a7655ab538 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:29.203755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b3f1-95fb-4de3-96cb-64a7655ab538 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:29.204551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932b3f1-95fb-4de3-96cb-64a7655ab538 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:29.206029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932b3f1-95fb-4de3-96cb-64a7655ab538 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:29.353522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 23/46] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:30:29 2022] POST /identity/v3/projects => generated 312 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:30:30.878375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87081e-8b14-4baf-bc23-aed5d0f3b334 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:30.878839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87081e-8b14-4baf-bc23-aed5d0f3b334 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:30.878839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87081e-8b14-4baf-bc23-aed5d0f3b334 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:30.880874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 24/47] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:30.886205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee5102-47e6-49ff-8995-43761b08ff84 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:30.886604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee5102-47e6-49ff-8995-43761b08ff84 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:30.886774 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee5102-47e6-49ff-8995-43761b08ff84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:30.905663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-63ee5102-47e6-49ff-8995-43761b08ff84 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:30.907237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-63ee5102-47e6-49ff-8995-43761b08ff84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:30.981544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 24/48] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:31.248894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87081e-8b14-4baf-bc23-aed5d0f3b334 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:31.310994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae243d-b507-4b1e-b873-581f51d7292c None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:31.311326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae243d-b507-4b1e-b873-581f51d7292c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:31.311572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae243d-b507-4b1e-b873-581f51d7292c None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:31.313234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 25/49] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:31.319646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70cc67-90c1-470d-8128-734bc54d06a1 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:31.319983 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70cc67-90c1-470d-8128-734bc54d06a1 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:31.320262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70cc67-90c1-470d-8128-734bc54d06a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:31.340020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1e70cc67-90c1-470d-8128-734bc54d06a1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:31.341408 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1e70cc67-90c1-470d-8128-734bc54d06a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:31.345906 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:31.423380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 26/50] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:31.466594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87081e-8b14-4baf-bc23-aed5d0f3b334 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:31.502013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc8cbd-f672-4578-bbe3-52d856f46b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:31.502582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc8cbd-f672-4578-bbe3-52d856f46b67 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:31.502869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc8cbd-f672-4578-bbe3-52d856f46b67 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:31.503148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc8cbd-f672-4578-bbe3-52d856f46b67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:31.515316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc8cbd-f672-4578-bbe3-52d856f46b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-dcbc8cbd-f672-4578-bbe3-52d856f46b67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:31.522448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:31.523328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:30:31.524175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 25/51] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:30:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:31.529817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70cc67-90c1-470d-8128-734bc54d06a1 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:31.658806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70cc67-90c1-470d-8128-734bc54d06a1 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:31.694713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31934b5-e9ec-4d99-b3c3-600a2e2cf404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:31.695336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31934b5-e9ec-4d99-b3c3-600a2e2cf404 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:31.695619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31934b5-e9ec-4d99-b3c3-600a2e2cf404 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:31.695912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31934b5-e9ec-4d99-b3c3-600a2e2cf404 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:31.696599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31934b5-e9ec-4d99-b3c3-600a2e2cf404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:31.696907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31934b5-e9ec-4d99-b3c3-600a2e2cf404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:31.698293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31934b5-e9ec-4d99-b3c3-600a2e2cf404 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:31.707216 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 27/52] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:31.713141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc8cbd-f672-4578-bbe3-52d856f46b67 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:31.716779 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:31.724241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31934b5-e9ec-4d99-b3c3-600a2e2cf404 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:31.727182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:31.829672 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc8cbd-f672-4578-bbe3-52d856f46b67 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:31.829992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31934b5-e9ec-4d99-b3c3-600a2e2cf404 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:31.867530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bf354-3662-4222-bbd6-13bceee424ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:31.868269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bf354-3662-4222-bbd6-13bceee424ca None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:31.868556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bf354-3662-4222-bbd6-13bceee424ca None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:31.868838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bf354-3662-4222-bbd6-13bceee424ca None admin] PATH_INFO: `/v3/users/3fea9e7c5b754acaa0233c15976ac3b3` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:31.868987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57fa92-a8e4-4a12-b977-808f381330a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:31.869628 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57fa92-a8e4-4a12-b977-808f381330a0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:31.869946 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57fa92-a8e4-4a12-b977-808f381330a0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:31.870129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bf354-3662-4222-bbd6-13bceee424ca None admin] RBAC: Authorizing `identity:get_user(user_id=3fea9e7c5b754acaa0233c15976ac3b3)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:31.870267 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57fa92-a8e4-4a12-b977-808f381330a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:31.871398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57fa92-a8e4-4a12-b977-808f381330a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:31.871598 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bf354-3662-4222-bbd6-13bceee424ca None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:31.872883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57fa92-a8e4-4a12-b977-808f381330a0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:31.873489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 26/53] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:31 2022] GET /identity/v3/users/3fea9e7c5b754acaa0233c15976ac3b3 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:31.874537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 28/54] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:31.878855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bf354-3662-4222-bbd6-13bceee424ca None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:31.880802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57fa92-a8e4-4a12-b977-808f381330a0 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:31.958439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:31.960800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:31.992921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bf354-3662-4222-bbd6-13bceee424ca None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:31.994132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57fa92-a8e4-4a12-b977-808f381330a0 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:32.033030 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d42687-79c7-4a72-8dc4-0ce80614f370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:32.033297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469b08f-d710-4b34-9d77-815f1e445726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:32.033849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d42687-79c7-4a72-8dc4-0ce80614f370 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:32.033950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469b08f-d710-4b34-9d77-815f1e445726 None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:32.034175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469b08f-d710-4b34-9d77-815f1e445726 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:32.034261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d42687-79c7-4a72-8dc4-0ce80614f370 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:32.034458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469b08f-d710-4b34-9d77-815f1e445726 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:32.034554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d42687-79c7-4a72-8dc4-0ce80614f370 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:32.035161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469b08f-d710-4b34-9d77-815f1e445726 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:32.036006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d42687-79c7-4a72-8dc4-0ce80614f370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:32.036780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469b08f-d710-4b34-9d77-815f1e445726 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:32.037592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d42687-79c7-4a72-8dc4-0ce80614f370 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:32.039265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 27/55] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:32.045081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d42687-79c7-4a72-8dc4-0ce80614f370 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:32.149433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d42687-79c7-4a72-8dc4-0ce80614f370 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:32.168334 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 29/56] 10.222.0.98 () {60 vars in 1140 bytes} [Fri Aug 26 15:30:31 2022] POST /identity/v3/projects => generated 298 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:30:32.191252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bf84e-66b1-4d5b-a031-e067b7aa3138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:32.192472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bf84e-66b1-4d5b-a031-e067b7aa3138 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:32.192759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bf84e-66b1-4d5b-a031-e067b7aa3138 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:32.193041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bf84e-66b1-4d5b-a031-e067b7aa3138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:32.193792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bf84e-66b1-4d5b-a031-e067b7aa3138 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.domain.id=default, user.id=3fea9e7c5b754acaa0233c15976ac3b3) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:32.194104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bf84e-66b1-4d5b-a031-e067b7aa3138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:32.195615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bf84e-66b1-4d5b-a031-e067b7aa3138 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:32.206806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 28/57] 10.222.0.98 () {56 vars in 1322 bytes} [Fri Aug 26 15:30:32 2022] GET /identity/v3/role_assignments?user.id=3fea9e7c5b754acaa0233c15976ac3b3&scope.domain.id=default&role.id=6ba5a5add2174983be37258aad1ee404 => generated 530 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:34.141708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28289-0a91-4287-9284-b1dc74e3a9da None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:34.142244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28289-0a91-4287-9284-b1dc74e3a9da None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:34.142319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28289-0a91-4287-9284-b1dc74e3a9da None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:34.143632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 30/58] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:34.149357 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3949a0-0b19-465c-ba38-25ffbba09937 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:34.149748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3949a0-0b19-465c-ba38-25ffbba09937 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:34.149931 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3949a0-0b19-465c-ba38-25ffbba09937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:34.156682 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:34.168573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-5a3949a0-0b19-465c-ba38-25ffbba09937 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:34.169790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5a3949a0-0b19-465c-ba38-25ffbba09937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:34.240070 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 29/59] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:34.514043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b28289-0a91-4287-9284-b1dc74e3a9da None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:34.623600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b28289-0a91-4287-9284-b1dc74e3a9da None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:34.648869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:34.660538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e9a2d-a7a3-43a1-a0e9-146fe825d6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:34.661272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9a2d-a7a3-43a1-a0e9-146fe825d6cb None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:34.661561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9a2d-a7a3-43a1-a0e9-146fe825d6cb None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:34.661838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9a2d-a7a3-43a1-a0e9-146fe825d6cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:34.663023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e9a2d-a7a3-43a1-a0e9-146fe825d6cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:34.664530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e9a2d-a7a3-43a1-a0e9-146fe825d6cb None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:34.666397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 31/60] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:34.671883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3949a0-0b19-465c-ba38-25ffbba09937 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:34.773144 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3949a0-0b19-465c-ba38-25ffbba09937 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:34.805004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:34.809666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5ff9d-e17c-4ee1-9363-94869f6d57db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:34.810307 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5ff9d-e17c-4ee1-9363-94869f6d57db None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:34.810597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5ff9d-e17c-4ee1-9363-94869f6d57db None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:34.810883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5ff9d-e17c-4ee1-9363-94869f6d57db None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:34.811564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5ff9d-e17c-4ee1-9363-94869f6d57db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:34.813094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5ff9d-e17c-4ee1-9363-94869f6d57db None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:34.946849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 30/61] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:30:34 2022] POST /identity/v3/users => generated 275 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:30:38.002088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4133b32-ead0-4955-9e83-cc2a1ebcbb36 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:38.002602 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4133b32-ead0-4955-9e83-cc2a1ebcbb36 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:38.002675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4133b32-ead0-4955-9e83-cc2a1ebcbb36 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:38.004081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 32/62] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:38.016054 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63278-ac6b-4d1c-8c59-c232ab6463c1 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:38.019966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63278-ac6b-4d1c-8c59-c232ab6463c1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:38.019966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63278-ac6b-4d1c-8c59-c232ab6463c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:38.044917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-d8c63278-ac6b-4d1c-8c59-c232ab6463c1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:38.046482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d8c63278-ac6b-4d1c-8c59-c232ab6463c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:38.135077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 31/63] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:38.157533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac9a91-d883-47b9-a2d7-c67bd4a085f6 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:38.157923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac9a91-d883-47b9-a2d7-c67bd4a085f6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:38.158285 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac9a91-d883-47b9-a2d7-c67bd4a085f6 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:38.159694 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 33/64] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:38.175617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83c2b2-dc4a-47db-8330-d7801c9eb99d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:38.175972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83c2b2-dc4a-47db-8330-d7801c9eb99d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:38.176244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83c2b2-dc4a-47db-8330-d7801c9eb99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:38.205057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-0a83c2b2-dc4a-47db-8330-d7801c9eb99d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:38.206279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0a83c2b2-dc4a-47db-8330-d7801c9eb99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:38.295056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 32/65] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:38.306887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047da38-1aa9-4ed6-855a-5bd488823e9e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:38.307469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047da38-1aa9-4ed6-855a-5bd488823e9e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:38.307977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047da38-1aa9-4ed6-855a-5bd488823e9e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:38.309768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 34/66] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:38.314904 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e982e-c8fb-4f57-8d6c-598bf0b7f345 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:38.315204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e982e-c8fb-4f57-8d6c-598bf0b7f345 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:38.315494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e982e-c8fb-4f57-8d6c-598bf0b7f345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:38.341527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-646e982e-c8fb-4f57-8d6c-598bf0b7f345 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:38.342746 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-646e982e-c8fb-4f57-8d6c-598bf0b7f345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:38.400664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:38.424685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 33/67] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:38.461452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047da38-1aa9-4ed6-855a-5bd488823e9e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:38.586764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dbf3c-b698-455b-b36d-7228d0953c54 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:38.587373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dbf3c-b698-455b-b36d-7228d0953c54 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:38.587943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dbf3c-b698-455b-b36d-7228d0953c54 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:38.589604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 34/68] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:30:38.594631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047da38-1aa9-4ed6-855a-5bd488823e9e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:38.601816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438d5a7-0a3a-448f-9182-135091cb7c08 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:38.602429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438d5a7-0a3a-448f-9182-135091cb7c08 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:38.602933 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438d5a7-0a3a-448f-9182-135091cb7c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:38.631585 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-3438d5a7-0a3a-448f-9182-135091cb7c08 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:38.634998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3438d5a7-0a3a-448f-9182-135091cb7c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:38.664521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7f1cd-0c14-4c1f-8bb2-30e684a5fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:38.665167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7f1cd-0c14-4c1f-8bb2-30e684a5fbe8 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:38.665455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7f1cd-0c14-4c1f-8bb2-30e684a5fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:38.665740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7f1cd-0c14-4c1f-8bb2-30e684a5fbe8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:38.677421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7f1cd-0c14-4c1f-8bb2-30e684a5fbe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-e0e7f1cd-0c14-4c1f-8bb2-30e684a5fbe8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:38.682448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:38.683350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:38.684183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 35/69] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:38.684831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7f1cd-0c14-4c1f-8bb2-30e684a5fbe8 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:38.719240 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 35/70] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:30:38.720459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438d5a7-0a3a-448f-9182-135091cb7c08 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:38.741301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:38.792430 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7f1cd-0c14-4c1f-8bb2-30e684a5fbe8 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:38.830541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed75d3c-c27d-4a78-9e09-d9325c9f7472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:38.831482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed75d3c-c27d-4a78-9e09-d9325c9f7472 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:38.832049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed75d3c-c27d-4a78-9e09-d9325c9f7472 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:38.832471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed75d3c-c27d-4a78-9e09-d9325c9f7472 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:38.846364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed75d3c-c27d-4a78-9e09-d9325c9f7472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-1ed75d3c-c27d-4a78-9e09-d9325c9f7472 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:38.851824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:38.852839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:30:38.853605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:38.855841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 36/71] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:30:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:38.860845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed75d3c-c27d-4a78-9e09-d9325c9f7472 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:38.862643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438d5a7-0a3a-448f-9182-135091cb7c08 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:38.875399 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:38.903817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d407a-d4b1-4f85-8b89-54076d849346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:38.904754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d407a-d4b1-4f85-8b89-54076d849346 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:38.905308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d407a-d4b1-4f85-8b89-54076d849346 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:38.905762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d407a-d4b1-4f85-8b89-54076d849346 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:38.906739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d407a-d4b1-4f85-8b89-54076d849346 None admin] RBAC: Adding query filter params (name=member) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:38.907299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d407a-d4b1-4f85-8b89-54076d849346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:38.908910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d407a-d4b1-4f85-8b89-54076d849346 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:38.922190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 36/72] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:30:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:38.923805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d407a-d4b1-4f85-8b89-54076d849346 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:38.994192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed75d3c-c27d-4a78-9e09-d9325c9f7472 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:39.008210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:39.035760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7200c-d137-41fc-b412-0aa566cbee5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:39.036494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7200c-d137-41fc-b412-0aa566cbee5b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.036796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7200c-d137-41fc-b412-0aa566cbee5b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.037235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7200c-d137-41fc-b412-0aa566cbee5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.038335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d407a-d4b1-4f85-8b89-54076d849346 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:39.049604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7200c-d137-41fc-b412-0aa566cbee5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-05c7200c-d137-41fc-b412-0aa566cbee5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:39.054922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:39.055796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:39.056659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 37/73] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:30:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:39.058053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7200c-d137-41fc-b412-0aa566cbee5b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:39.083515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe32f88-961a-4f62-964f-faa2b46c5251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:39.084220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe32f88-961a-4f62-964f-faa2b46c5251 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.084505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe32f88-961a-4f62-964f-faa2b46c5251 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.084790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe32f88-961a-4f62-964f-faa2b46c5251 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.085563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe32f88-961a-4f62-964f-faa2b46c5251 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:39.085877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe32f88-961a-4f62-964f-faa2b46c5251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:39.087493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe32f88-961a-4f62-964f-faa2b46c5251 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:39.095919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 37/74] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:30:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:39.096788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe32f88-961a-4f62-964f-faa2b46c5251 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:39.127646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:39.163712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7200c-d137-41fc-b412-0aa566cbee5b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:39.197629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe32f88-961a-4f62-964f-faa2b46c5251 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:39.200110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c3d49-db22-465b-a8d1-cab5cd17152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:39.200758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c3d49-db22-465b-a8d1-cab5cd17152f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.201042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c3d49-db22-465b-a8d1-cab5cd17152f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.201329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c3d49-db22-465b-a8d1-cab5cd17152f None admin] PATH_INFO: `/v3/users/99da68de463743fdb9293f16abb81a3b` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.216148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c3d49-db22-465b-a8d1-cab5cd17152f None admin] RBAC: Authorizing `identity:get_user(user_id=99da68de463743fdb9293f16abb81a3b)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:39.217643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c3d49-db22-465b-a8d1-cab5cd17152f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:39.219319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 38/75] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:39 2022] GET /identity/v3/users/99da68de463743fdb9293f16abb81a3b => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:39.225264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c3d49-db22-465b-a8d1-cab5cd17152f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:39.231942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa6807-80aa-4859-ba07-8eb20f456b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:39.232589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa6807-80aa-4859-ba07-8eb20f456b89 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.232868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa6807-80aa-4859-ba07-8eb20f456b89 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.233142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa6807-80aa-4859-ba07-8eb20f456b89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.244024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa6807-80aa-4859-ba07-8eb20f456b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-52fa6807-80aa-4859-ba07-8eb20f456b89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:39.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:39.249653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:30:39.250482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 38/76] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:39.252339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db1b66-a078-4cdb-b38d-d0eb474693e5 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.252635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db1b66-a078-4cdb-b38d-d0eb474693e5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.252918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db1b66-a078-4cdb-b38d-d0eb474693e5 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.254181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 39/77] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:39.254914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db1b66-a078-4cdb-b38d-d0eb474693e5 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:39.257391 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:39.328410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c3d49-db22-465b-a8d1-cab5cd17152f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:39.346458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:39.356887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db1b66-a078-4cdb-b38d-d0eb474693e5 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:39.362149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fdfd6-fd94-4ecc-bf4f-433426187283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:39.362763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fdfd6-fd94-4ecc-bf4f-433426187283 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.363103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fdfd6-fd94-4ecc-bf4f-433426187283 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.363391 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fdfd6-fd94-4ecc-bf4f-433426187283 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.364095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fdfd6-fd94-4ecc-bf4f-433426187283 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:39.364971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fdfd6-fd94-4ecc-bf4f-433426187283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:39.366362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fdfd6-fd94-4ecc-bf4f-433426187283 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:39.374481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 39/78] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:39.375201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fdfd6-fd94-4ecc-bf4f-433426187283 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:39.395822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcd54e-8c29-4783-bcb1-f8ce3d39b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:39.396498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcd54e-8c29-4783-bcb1-f8ce3d39b7ed None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.396787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcd54e-8c29-4783-bcb1-f8ce3d39b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.397071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcd54e-8c29-4783-bcb1-f8ce3d39b7ed None admin] PATH_INFO: `/v3/users/99da68de463743fdb9293f16abb81a3b` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.398223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcd54e-8c29-4783-bcb1-f8ce3d39b7ed None admin] RBAC: Authorizing `identity:get_user(user_id=99da68de463743fdb9293f16abb81a3b)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:39.399694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcd54e-8c29-4783-bcb1-f8ce3d39b7ed None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:39.401324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 40/79] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:39 2022] GET /identity/v3/users/99da68de463743fdb9293f16abb81a3b => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:39.401324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcd54e-8c29-4783-bcb1-f8ce3d39b7ed None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:39.469733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:39.479270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fdfd6-fd94-4ecc-bf4f-433426187283 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:39.501762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcd54e-8c29-4783-bcb1-f8ce3d39b7ed None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:39.514385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda77e9b-b9c9-4619-b0d1-c6eb48ecc08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:39.515001 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda77e9b-b9c9-4619-b0d1-c6eb48ecc08f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.515291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda77e9b-b9c9-4619-b0d1-c6eb48ecc08f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.515578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda77e9b-b9c9-4619-b0d1-c6eb48ecc08f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.534698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f739010-c7d1-483e-893c-9b79ecf35d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:39.535374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f739010-c7d1-483e-893c-9b79ecf35d9d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.535590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f739010-c7d1-483e-893c-9b79ecf35d9d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.535888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f739010-c7d1-483e-893c-9b79ecf35d9d None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.537069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f739010-c7d1-483e-893c-9b79ecf35d9d None admin] RBAC: Adding query filter params (name=member) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:39.537381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f739010-c7d1-483e-893c-9b79ecf35d9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:39.538739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f739010-c7d1-483e-893c-9b79ecf35d9d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:39.546540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 41/80] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:30:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:39.547657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6140fa-7361-42c6-865b-4abf7f34ad32 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.547973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6140fa-7361-42c6-865b-4abf7f34ad32 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.548252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6140fa-7361-42c6-865b-4abf7f34ad32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.566402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-5c6140fa-7361-42c6-865b-4abf7f34ad32 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:39.567630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5c6140fa-7361-42c6-865b-4abf7f34ad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:39.584822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:39.586019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda77e9b-b9c9-4619-b0d1-c6eb48ecc08f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:39.587515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda77e9b-b9c9-4619-b0d1-c6eb48ecc08f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:39.635989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 42/81] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:39.637188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6140fa-7361-42c6-865b-4abf7f34ad32 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:39.693474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:39.703744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 40/82] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:39.704743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda77e9b-b9c9-4619-b0d1-c6eb48ecc08f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:39.752716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6140fa-7361-42c6-865b-4abf7f34ad32 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:39.792563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570890a-6fdf-4aad-9152-b367c27a9f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:39.793203 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570890a-6fdf-4aad-9152-b367c27a9f13 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.793494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570890a-6fdf-4aad-9152-b367c27a9f13 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.793775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570890a-6fdf-4aad-9152-b367c27a9f13 None admin] PATH_INFO: `/v3/users/3fea9e7c5b754acaa0233c15976ac3b3` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.794899 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570890a-6fdf-4aad-9152-b367c27a9f13 None admin] RBAC: Authorizing `identity:get_user(user_id=3fea9e7c5b754acaa0233c15976ac3b3)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:39.796377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570890a-6fdf-4aad-9152-b367c27a9f13 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:39.798146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 43/83] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:39 2022] GET /identity/v3/users/3fea9e7c5b754acaa0233c15976ac3b3 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:39.799107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570890a-6fdf-4aad-9152-b367c27a9f13 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:39.821663 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:39.821828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda77e9b-b9c9-4619-b0d1-c6eb48ecc08f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:39.872805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3c767-2b5a-4694-a8c9-070163c08ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:39.875763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3c767-2b5a-4694-a8c9-070163c08ac4 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.875763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3c767-2b5a-4694-a8c9-070163c08ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.875763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3c767-2b5a-4694-a8c9-070163c08ac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.916802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570890a-6fdf-4aad-9152-b367c27a9f13 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:39.937239 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:39.947409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3c767-2b5a-4694-a8c9-070163c08ac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:39.948974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3c767-2b5a-4694-a8c9-070163c08ac4 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:39.951344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265510dd-eb49-4a08-82af-c5aa79bd726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:39.951639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265510dd-eb49-4a08-82af-c5aa79bd726e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:39.951961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265510dd-eb49-4a08-82af-c5aa79bd726e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:39.952249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265510dd-eb49-4a08-82af-c5aa79bd726e None admin] PATH_INFO: `/v3/users/99da68de463743fdb9293f16abb81a3b` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:39.953388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265510dd-eb49-4a08-82af-c5aa79bd726e None admin] RBAC: Authorizing `identity:get_user(user_id=99da68de463743fdb9293f16abb81a3b)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:39.954862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265510dd-eb49-4a08-82af-c5aa79bd726e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:39.956710 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 44/84] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:39 2022] GET /identity/v3/users/99da68de463743fdb9293f16abb81a3b => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:39.957374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265510dd-eb49-4a08-82af-c5aa79bd726e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:40.045187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:40.048625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 41/85] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:40.051889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3c767-2b5a-4694-a8c9-070163c08ac4 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:40.061166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265510dd-eb49-4a08-82af-c5aa79bd726e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:40.097707 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dcbc5-4237-4802-a995-c803df6de220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:40.098360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dcbc5-4237-4802-a995-c803df6de220 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:40.098701 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dcbc5-4237-4802-a995-c803df6de220 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:40.099042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dcbc5-4237-4802-a995-c803df6de220 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:40.100349 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795dcbc5-4237-4802-a995-c803df6de220 None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:40.101811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795dcbc5-4237-4802-a995-c803df6de220 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:40.103444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 45/86] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:39 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:40.104243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dcbc5-4237-4802-a995-c803df6de220 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:40.146892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:40.156112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3c767-2b5a-4694-a8c9-070163c08ac4 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:40.189655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d495bb1-6eca-43c2-8995-9d1a3a2d0d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:40.190266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d495bb1-6eca-43c2-8995-9d1a3a2d0d68 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:40.190558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d495bb1-6eca-43c2-8995-9d1a3a2d0d68 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:40.190845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d495bb1-6eca-43c2-8995-9d1a3a2d0d68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:40.207350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dcbc5-4237-4802-a995-c803df6de220 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:40.241106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c363b-df11-4cf0-8df1-5577ebf69a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:40.241734 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c363b-df11-4cf0-8df1-5577ebf69a47 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:40.242018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c363b-df11-4cf0-8df1-5577ebf69a47 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:40.242294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c363b-df11-4cf0-8df1-5577ebf69a47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:40.243402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c363b-df11-4cf0-8df1-5577ebf69a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:40.244871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c363b-df11-4cf0-8df1-5577ebf69a47 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:40.246415 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 46/87] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:40.247180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c363b-df11-4cf0-8df1-5577ebf69a47 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:40.260066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d495bb1-6eca-43c2-8995-9d1a3a2d0d68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:40.261706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d495bb1-6eca-43c2-8995-9d1a3a2d0d68 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:40.273568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:40.348089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c363b-df11-4cf0-8df1-5577ebf69a47 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:40.349792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:40.360531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 42/88] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:40.361129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d495bb1-6eca-43c2-8995-9d1a3a2d0d68 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:40.383840 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a81f8-95cf-4a21-84cb-f2ada6b8a593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:40.384484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a81f8-95cf-4a21-84cb-f2ada6b8a593 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:40.384885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a81f8-95cf-4a21-84cb-f2ada6b8a593 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:40.385176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a81f8-95cf-4a21-84cb-f2ada6b8a593 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:40.454007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a81f8-95cf-4a21-84cb-f2ada6b8a593 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:40.455396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a81f8-95cf-4a21-84cb-f2ada6b8a593 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:40.462806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d495bb1-6eca-43c2-8995-9d1a3a2d0d68 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:40.481950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:40.496278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f5a99-9e85-4067-a929-b3d22d79b4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:40.500747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f5a99-9e85-4067-a929-b3d22d79b4d8 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:40.500747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f5a99-9e85-4067-a929-b3d22d79b4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:40.500747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f5a99-9e85-4067-a929-b3d22d79b4d8 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:40.500747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f5a99-9e85-4067-a929-b3d22d79b4d8 None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:40.501159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f5a99-9e85-4067-a929-b3d22d79b4d8 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:40.502969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 43/89] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:40 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:40.503912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f5a99-9e85-4067-a929-b3d22d79b4d8 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:40.562520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 47/90] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:40.562520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a81f8-95cf-4a21-84cb-f2ada6b8a593 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:40.563976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:40.605083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f5a99-9e85-4067-a929-b3d22d79b4d8 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:40.638386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962ac81-4c67-4434-a5bb-07f05ce68b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:40.638995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962ac81-4c67-4434-a5bb-07f05ce68b66 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:40.639284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962ac81-4c67-4434-a5bb-07f05ce68b66 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:40.639564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962ac81-4c67-4434-a5bb-07f05ce68b66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:40.640331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962ac81-4c67-4434-a5bb-07f05ce68b66 None admin] RBAC: Adding query filter params (role.id=0166494127b14615900b2c121908b2ba, scope.project.id=f1c8960e806347658b410f7278fe7ffd, user.id=99da68de463743fdb9293f16abb81a3b) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:40.640649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962ac81-4c67-4434-a5bb-07f05ce68b66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:40.642116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962ac81-4c67-4434-a5bb-07f05ce68b66 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:40.648098 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 44/91] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:40 2022] GET /identity/v3/role_assignments?user.id=99da68de463743fdb9293f16abb81a3b&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=0166494127b14615900b2c121908b2ba => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:40.648933 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962ac81-4c67-4434-a5bb-07f05ce68b66 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:40.660500 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a81f8-95cf-4a21-84cb-f2ada6b8a593 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:40.687099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:40.703300 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74068bca-a797-43dc-9cf8-6663e2c9ab54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:40.704098 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068bca-a797-43dc-9cf8-6663e2c9ab54 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:40.704327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068bca-a797-43dc-9cf8-6663e2c9ab54 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:40.704612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068bca-a797-43dc-9cf8-6663e2c9ab54 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:40.705282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74068bca-a797-43dc-9cf8-6663e2c9ab54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:40.706782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74068bca-a797-43dc-9cf8-6663e2c9ab54 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:40.764209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962ac81-4c67-4434-a5bb-07f05ce68b66 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:40.788385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:40.800194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b301dc-943a-4958-af00-41eacbd6f467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:40.800828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b301dc-943a-4958-af00-41eacbd6f467 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:40.801119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b301dc-943a-4958-af00-41eacbd6f467 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:40.801422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b301dc-943a-4958-af00-41eacbd6f467 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:40.802549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b301dc-943a-4958-af00-41eacbd6f467 None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:40.804054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b301dc-943a-4958-af00-41eacbd6f467 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:40.805682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 45/92] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:40 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:40.806523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b301dc-943a-4958-af00-41eacbd6f467 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:40.844553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 48/93] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:30:40 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:30:40.845677 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74068bca-a797-43dc-9cf8-6663e2c9ab54 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:40.921769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b301dc-943a-4958-af00-41eacbd6f467 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:40.961929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c16728-c5ff-4a03-812a-099392586344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:40.962994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c16728-c5ff-4a03-812a-099392586344 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:40.962994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c16728-c5ff-4a03-812a-099392586344 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:40.963193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c16728-c5ff-4a03-812a-099392586344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:40.963936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c16728-c5ff-4a03-812a-099392586344 None admin] RBAC: Adding query filter params (role.id=e0822035b245457eade1325bbe888913, scope.project.id=f1c8960e806347658b410f7278fe7ffd, user.id=99da68de463743fdb9293f16abb81a3b) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:40.964259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c16728-c5ff-4a03-812a-099392586344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:40.966159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c16728-c5ff-4a03-812a-099392586344 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:40.972553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 46/94] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:40 2022] GET /identity/v3/role_assignments?user.id=99da68de463743fdb9293f16abb81a3b&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=e0822035b245457eade1325bbe888913 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:40.973270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c16728-c5ff-4a03-812a-099392586344 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:40.998878 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74068bca-a797-43dc-9cf8-6663e2c9ab54 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:41.047648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372e4ae-850d-40a6-9f2e-1e339af4ceae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:41.049286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372e4ae-850d-40a6-9f2e-1e339af4ceae None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:41.049957 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372e4ae-850d-40a6-9f2e-1e339af4ceae None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:41.050442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372e4ae-850d-40a6-9f2e-1e339af4ceae None admin] PATH_INFO: `/v3/projects/5f8cbb25889d4713a0d84901578c0cf8` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:41.052041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372e4ae-850d-40a6-9f2e-1e339af4ceae None admin] RBAC: Authorizing `identity:get_project(project_id=5f8cbb25889d4713a0d84901578c0cf8)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:41.053706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372e4ae-850d-40a6-9f2e-1e339af4ceae None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:41.055681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 49/95] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:40 2022] GET /identity/v3/projects/5f8cbb25889d4713a0d84901578c0cf8 => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:41.060807 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:41.062873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372e4ae-850d-40a6-9f2e-1e339af4ceae None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:41.127131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:41.130462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c16728-c5ff-4a03-812a-099392586344 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:41.176658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce5111-8abd-453e-99f0-3bb021574ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:41.177707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce5111-8abd-453e-99f0-3bb021574ef9 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:41.178192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce5111-8abd-453e-99f0-3bb021574ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:41.178667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce5111-8abd-453e-99f0-3bb021574ef9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:41.179594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce5111-8abd-453e-99f0-3bb021574ef9 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=f1c8960e806347658b410f7278fe7ffd, user.id=3fea9e7c5b754acaa0233c15976ac3b3) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:41.180128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce5111-8abd-453e-99f0-3bb021574ef9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:41.182998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce5111-8abd-453e-99f0-3bb021574ef9 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:41.189359 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372e4ae-850d-40a6-9f2e-1e339af4ceae None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:41.190920 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 47/96] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:40 2022] GET /identity/v3/role_assignments?user.id=3fea9e7c5b754acaa0233c15976ac3b3&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=6ba5a5add2174983be37258aad1ee404 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:41.232456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e52be5-1501-4643-94c2-3a12ca2f15e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:41.233498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e52be5-1501-4643-94c2-3a12ca2f15e8 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:41.234187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e52be5-1501-4643-94c2-3a12ca2f15e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:41.234665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e52be5-1501-4643-94c2-3a12ca2f15e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:41.235672 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e52be5-1501-4643-94c2-3a12ca2f15e8 None admin] RBAC: Adding query filter params (role.id=0166494127b14615900b2c121908b2ba, scope.project.id=5f8cbb25889d4713a0d84901578c0cf8, user.id=99da68de463743fdb9293f16abb81a3b) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:41.236221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e52be5-1501-4643-94c2-3a12ca2f15e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:41.237980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e52be5-1501-4643-94c2-3a12ca2f15e8 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:41.245544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 50/97] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:41 2022] GET /identity/v3/role_assignments?user.id=99da68de463743fdb9293f16abb81a3b&scope.project.id=5f8cbb25889d4713a0d84901578c0cf8&role.id=0166494127b14615900b2c121908b2ba => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:42.989767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddf4a8-2692-4253-8855-257308cd9457 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:42.990867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddf4a8-2692-4253-8855-257308cd9457 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:42.991408 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddf4a8-2692-4253-8855-257308cd9457 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:42.993180 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 48/98] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:42.999782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0badb-9b2d-47c7-86be-3a8ebd658722 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:43.000781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0badb-9b2d-47c7-86be-3a8ebd658722 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:43.001282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0badb-9b2d-47c7-86be-3a8ebd658722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:43.028945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-31a0badb-9b2d-47c7-86be-3a8ebd658722 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:43.030622 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-31a0badb-9b2d-47c7-86be-3a8ebd658722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:43.133540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 51/99] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:43.306816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e0400-733c-446f-88cd-1fb7884e0a41 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:43.307152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e0400-733c-446f-88cd-1fb7884e0a41 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:43.307335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e0400-733c-446f-88cd-1fb7884e0a41 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:43.308846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 49/100] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:43.314031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa2ff4-3968-4e96-9651-3601809e29cc None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:43.314332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa2ff4-3968-4e96-9651-3601809e29cc None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:43.314625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa2ff4-3968-4e96-9651-3601809e29cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:43.335299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-a0fa2ff4-3968-4e96-9651-3601809e29cc None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:43.336580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a0fa2ff4-3968-4e96-9651-3601809e29cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:43.391561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:43.432519 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 52/101] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:30:43.660996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e0400-733c-446f-88cd-1fb7884e0a41 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:43.793284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e0400-733c-446f-88cd-1fb7884e0a41 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:43.819153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:43.843484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32678c-acc9-40a4-8798-53754f6b1e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:43.844168 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32678c-acc9-40a4-8798-53754f6b1e6b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:43.844457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32678c-acc9-40a4-8798-53754f6b1e6b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:43.844742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32678c-acc9-40a4-8798-53754f6b1e6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:43.857402 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32678c-acc9-40a4-8798-53754f6b1e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-0a32678c-acc9-40a4-8798-53754f6b1e6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:43.862864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:43.863803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:43.864702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:43.867090 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 50/102] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:30:43.873360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa2ff4-3968-4e96-9651-3601809e29cc None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:44.002279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa2ff4-3968-4e96-9651-3601809e29cc None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:44.041573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec3cec-8653-4836-94a3-478f0e6c245d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:44.042639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec3cec-8653-4836-94a3-478f0e6c245d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.043138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec3cec-8653-4836-94a3-478f0e6c245d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.043618 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec3cec-8653-4836-94a3-478f0e6c245d None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.044650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec3cec-8653-4836-94a3-478f0e6c245d None admin] RBAC: Adding query filter params (name=member) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:44.045196 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec3cec-8653-4836-94a3-478f0e6c245d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:44.046850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec3cec-8653-4836-94a3-478f0e6c245d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:44.057084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 53/103] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:30:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:44.065302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32678c-acc9-40a4-8798-53754f6b1e6b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:44.077943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec3cec-8653-4836-94a3-478f0e6c245d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:44.140677 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:44.192964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32678c-acc9-40a4-8798-53754f6b1e6b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:44.236701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369eb60-b788-4ce7-9117-081ada0c3ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:44.237730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369eb60-b788-4ce7-9117-081ada0c3ba7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.238740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369eb60-b788-4ce7-9117-081ada0c3ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.239238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369eb60-b788-4ce7-9117-081ada0c3ba7 None admin] PATH_INFO: `/v3/users/99da68de463743fdb9293f16abb81a3b` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.241656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369eb60-b788-4ce7-9117-081ada0c3ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=99da68de463743fdb9293f16abb81a3b)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:44.243449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369eb60-b788-4ce7-9117-081ada0c3ba7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:44.246020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 51/104] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:44 2022] GET /identity/v3/users/99da68de463743fdb9293f16abb81a3b => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:44.253677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369eb60-b788-4ce7-9117-081ada0c3ba7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:44.290792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec3cec-8653-4836-94a3-478f0e6c245d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:44.320655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:44.370177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440ea4f-1250-4f75-b8aa-e900db2e588b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:44.371034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440ea4f-1250-4f75-b8aa-e900db2e588b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.371161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440ea4f-1250-4f75-b8aa-e900db2e588b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.371409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440ea4f-1250-4f75-b8aa-e900db2e588b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.382263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440ea4f-1250-4f75-b8aa-e900db2e588b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-1440ea4f-1250-4f75-b8aa-e900db2e588b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:44.387018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:44.387940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:30:44.388747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 54/105] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:30:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:44.390115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5019159-e07d-4867-8556-32971e3fe5ce None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.390115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5019159-e07d-4867-8556-32971e3fe5ce None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.390401 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5019159-e07d-4867-8556-32971e3fe5ce None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.391808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 55/106] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:44.392690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369eb60-b788-4ce7-9117-081ada0c3ba7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:44.392896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefa133-13a7-48a8-970f-5e94d921da7a None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.393159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefa133-13a7-48a8-970f-5e94d921da7a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.398861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefa133-13a7-48a8-970f-5e94d921da7a None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.398861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 56/107] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:44.398861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefa133-13a7-48a8-970f-5e94d921da7a None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:44.436604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbfe07-cc5c-4659-9f5e-c57409ee88b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:44.437252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbfe07-cc5c-4659-9f5e-c57409ee88b1 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.437542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbfe07-cc5c-4659-9f5e-c57409ee88b1 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.437825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbfe07-cc5c-4659-9f5e-c57409ee88b1 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.438965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbfe07-cc5c-4659-9f5e-c57409ee88b1 None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:44.440491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbfe07-cc5c-4659-9f5e-c57409ee88b1 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:44.442161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 52/108] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:44 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:44.443980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419a61a-3095-4cf4-9dcb-ecdefba089c3 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.444274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419a61a-3095-4cf4-9dcb-ecdefba089c3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.444556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419a61a-3095-4cf4-9dcb-ecdefba089c3 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.445789 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 53/109] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:44.446818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf1fb-426f-4663-bd01-6f5da0cb450f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.447106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf1fb-426f-4663-bd01-6f5da0cb450f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.448127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf1fb-426f-4663-bd01-6f5da0cb450f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.466845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-03ecf1fb-426f-4663-bd01-6f5da0cb450f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:44.468097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-03ecf1fb-426f-4663-bd01-6f5da0cb450f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:44.511584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:44.520454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefa133-13a7-48a8-970f-5e94d921da7a None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:44.542875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 54/110] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:44.551919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c695ce8-f044-4919-9288-72257730825a None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.551919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c695ce8-f044-4919-9288-72257730825a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.551919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c695ce8-f044-4919-9288-72257730825a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.565305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-8c695ce8-f044-4919-9288-72257730825a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:44.566656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8c695ce8-f044-4919-9288-72257730825a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:44.569768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7ac81-2ef9-4660-bed2-db09c2677459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:44.571525 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7ac81-2ef9-4660-bed2-db09c2677459 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.572139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7ac81-2ef9-4660-bed2-db09c2677459 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.572584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7ac81-2ef9-4660-bed2-db09c2677459 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.573418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7ac81-2ef9-4660-bed2-db09c2677459 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:44.573873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7ac81-2ef9-4660-bed2-db09c2677459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:44.575419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7ac81-2ef9-4660-bed2-db09c2677459 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:44.587161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 57/111] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:30:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:44.588321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7ac81-2ef9-4660-bed2-db09c2677459 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:44.623748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:44.650698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 55/112] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:44.652208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33bfd4-f28e-46b6-976a-0dace52a7595 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.652506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33bfd4-f28e-46b6-976a-0dace52a7595 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.652788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33bfd4-f28e-46b6-976a-0dace52a7595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.673185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-8b33bfd4-f28e-46b6-976a-0dace52a7595 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:44.674683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8b33bfd4-f28e-46b6-976a-0dace52a7595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:44.722480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7ac81-2ef9-4660-bed2-db09c2677459 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:44.753871 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 56/113] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:44.755449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33bfd4-f28e-46b6-976a-0dace52a7595 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:44.765025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ac7e3-3189-45b0-aa14-7627af689cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:44.765946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ac7e3-3189-45b0-aa14-7627af689cc9 None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.766377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ac7e3-3189-45b0-aa14-7627af689cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.766820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ac7e3-3189-45b0-aa14-7627af689cc9 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd/users/99da68de463743fdb9293f16abb81a3b/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.770009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ac7e3-3189-45b0-aa14-7627af689cc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1c8960e806347658b410f7278fe7ffd, user_id=99da68de463743fdb9293f16abb81a3b, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:44.771834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ac7e3-3189-45b0-aa14-7627af689cc9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:44.859321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 58/114] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:30:44 2022] PUT /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd/users/99da68de463743fdb9293f16abb81a3b/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:30:44.861253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf0f8-0f33-45c3-ba2b-757c154a4701 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.861551 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf0f8-0f33-45c3-ba2b-757c154a4701 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.861842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf0f8-0f33-45c3-ba2b-757c154a4701 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.864155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 59/115] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:44.868953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcf0f8-0f33-45c3-ba2b-757c154a4701 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:44.876861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33bfd4-f28e-46b6-976a-0dace52a7595 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:44.880652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:44.896484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:44.939164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07057b4-678c-4590-b031-1798dd9be88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:44.939842 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07057b4-678c-4590-b031-1798dd9be88e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.948730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07057b4-678c-4590-b031-1798dd9be88e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.948730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07057b4-678c-4590-b031-1798dd9be88e None admin] PATH_INFO: `/v3/users/99da68de463743fdb9293f16abb81a3b` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.948730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07057b4-678c-4590-b031-1798dd9be88e None admin] RBAC: Authorizing `identity:get_user(user_id=99da68de463743fdb9293f16abb81a3b)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:44.948730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07057b4-678c-4590-b031-1798dd9be88e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:44.948730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 57/116] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:44 2022] GET /identity/v3/users/99da68de463743fdb9293f16abb81a3b => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:44.952160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b6b54-e774-411d-afd9-d96adf381e8d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:44.952441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b6b54-e774-411d-afd9-d96adf381e8d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:44.952732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b6b54-e774-411d-afd9-d96adf381e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:44.996779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-474b6b54-e774-411d-afd9-d96adf381e8d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:44.998024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-474b6b54-e774-411d-afd9-d96adf381e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:45.063742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcf0f8-0f33-45c3-ba2b-757c154a4701 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:45.085441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 58/117] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:45.086415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b6b54-e774-411d-afd9-d96adf381e8d None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:45.104017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75cdd6-0fea-4897-8778-f950a0e0b9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:45.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75cdd6-0fea-4897-8778-f950a0e0b9f0 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:45.105358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75cdd6-0fea-4897-8778-f950a0e0b9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:45.105805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75cdd6-0fea-4897-8778-f950a0e0b9f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:45.107136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75cdd6-0fea-4897-8778-f950a0e0b9f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:45.108742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75cdd6-0fea-4897-8778-f950a0e0b9f0 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:45.114333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 60/118] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:45.114333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75cdd6-0fea-4897-8778-f950a0e0b9f0 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:45.218637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:45.226270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b6b54-e774-411d-afd9-d96adf381e8d None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:45.234837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75cdd6-0fea-4897-8778-f950a0e0b9f0 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:45.240511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:45.277191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c3429-b25b-4e9f-bc83-53081a0d436b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:45.278786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c3429-b25b-4e9f-bc83-53081a0d436b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:45.279075 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c3429-b25b-4e9f-bc83-53081a0d436b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:45.279830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c3429-b25b-4e9f-bc83-53081a0d436b None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:45.282269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e60948-908f-4669-acb5-b48d4fdc3a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:45.283110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c3429-b25b-4e9f-bc83-53081a0d436b None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:45.284107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e60948-908f-4669-acb5-b48d4fdc3a51 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:45.284601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c3429-b25b-4e9f-bc83-53081a0d436b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:45.284835 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e60948-908f-4669-acb5-b48d4fdc3a51 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:45.285310 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e60948-908f-4669-acb5-b48d4fdc3a51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:45.286256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 59/119] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:45 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:45.287639 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c3429-b25b-4e9f-bc83-53081a0d436b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:45.306470 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e60948-908f-4669-acb5-b48d4fdc3a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-08e60948-908f-4669-acb5-b48d4fdc3a51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:45.313326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:45.314233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:45.315024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:30:45.318392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 61/120] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:45.319398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e60948-908f-4669-acb5-b48d4fdc3a51 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:45.411023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c3429-b25b-4e9f-bc83-53081a0d436b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:45.438864 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e60948-908f-4669-acb5-b48d4fdc3a51 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:45.450721 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238e625-a7c7-40d1-8564-56277b51d44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:45.451366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238e625-a7c7-40d1-8564-56277b51d44f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:45.451657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238e625-a7c7-40d1-8564-56277b51d44f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:45.452052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238e625-a7c7-40d1-8564-56277b51d44f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:45.463993 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238e625-a7c7-40d1-8564-56277b51d44f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-f238e625-a7c7-40d1-8564-56277b51d44f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:45.469980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:45.471081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 60/121] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:30:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:45.472353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238e625-a7c7-40d1-8564-56277b51d44f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:45.480530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ff1f4-9cec-4c27-8c2b-2fd243b98ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:45.481163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ff1f4-9cec-4c27-8c2b-2fd243b98ac9 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:45.481452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ff1f4-9cec-4c27-8c2b-2fd243b98ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:45.481731 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ff1f4-9cec-4c27-8c2b-2fd243b98ac9 None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:45.482384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ff1f4-9cec-4c27-8c2b-2fd243b98ac9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:45.483892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ff1f4-9cec-4c27-8c2b-2fd243b98ac9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:45.501856 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:45.507596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:45.578492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238e625-a7c7-40d1-8564-56277b51d44f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:45.608626 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 62/122] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:30:45 2022] POST /identity/v3/projects => generated 302 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:30:45.609699 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ff1f4-9cec-4c27-8c2b-2fd243b98ac9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:45.624086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c0120-e127-4faf-8102-2154daafe217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:45.625808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c0120-e127-4faf-8102-2154daafe217 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:45.626278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c0120-e127-4faf-8102-2154daafe217 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:45.626734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c0120-e127-4faf-8102-2154daafe217 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd/users/99da68de463743fdb9293f16abb81a3b/roles/e0822035b245457eade1325bbe888913` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:45.630581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c0120-e127-4faf-8102-2154daafe217 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1c8960e806347658b410f7278fe7ffd, user_id=99da68de463743fdb9293f16abb81a3b, role_id=e0822035b245457eade1325bbe888913)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:45.633236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c0120-e127-4faf-8102-2154daafe217 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:45.714871 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 61/123] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:30:45 2022] PUT /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd/users/99da68de463743fdb9293f16abb81a3b/roles/e0822035b245457eade1325bbe888913 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:30:45.722001 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c0120-e127-4faf-8102-2154daafe217 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:45.758601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ff1f4-9cec-4c27-8c2b-2fd243b98ac9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:45.809575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f8252-7a62-42ea-a12f-02fd8405c2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:45.810463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f8252-7a62-42ea-a12f-02fd8405c2ed None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:45.810898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f8252-7a62-42ea-a12f-02fd8405c2ed None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:45.811280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f8252-7a62-42ea-a12f-02fd8405c2ed None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:45.812328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f8252-7a62-42ea-a12f-02fd8405c2ed None admin] RBAC: Adding query filter params (name=member) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:45.813122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f8252-7a62-42ea-a12f-02fd8405c2ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:45.814676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f8252-7a62-42ea-a12f-02fd8405c2ed None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:45.827562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 63/124] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:30:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:45.828843 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f8252-7a62-42ea-a12f-02fd8405c2ed None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:45.875737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:45.886669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c0120-e127-4faf-8102-2154daafe217 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:45.928869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:45.940558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90525e-45de-4952-9071-ecc1f672d9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:45.941507 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90525e-45de-4952-9071-ecc1f672d9c7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:45.942009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90525e-45de-4952-9071-ecc1f672d9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:45.942446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90525e-45de-4952-9071-ecc1f672d9c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:45.957960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90525e-45de-4952-9071-ecc1f672d9c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-ed90525e-45de-4952-9071-ecc1f672d9c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:45.964678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:30:45.966436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:45.969074 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 62/125] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:45.970071 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90525e-45de-4952-9071-ecc1f672d9c7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:45.994013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f8252-7a62-42ea-a12f-02fd8405c2ed None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:46.092342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90525e-45de-4952-9071-ecc1f672d9c7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:46.102814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca8c51-8634-42ec-8070-8915e4cb9feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:46.104204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca8c51-8634-42ec-8070-8915e4cb9feb None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:46.104669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca8c51-8634-42ec-8070-8915e4cb9feb None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:46.105193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca8c51-8634-42ec-8070-8915e4cb9feb None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:46.106104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca8c51-8634-42ec-8070-8915e4cb9feb None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:46.106587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca8c51-8634-42ec-8070-8915e4cb9feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:46.108210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca8c51-8634-42ec-8070-8915e4cb9feb None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:46.125756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 64/126] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:46.126970 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca8c51-8634-42ec-8070-8915e4cb9feb None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:46.168423 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6af63b-d22e-4531-a2d5-b6492f9216b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:46.169511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6af63b-d22e-4531-a2d5-b6492f9216b3 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:46.169976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6af63b-d22e-4531-a2d5-b6492f9216b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:46.170432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6af63b-d22e-4531-a2d5-b6492f9216b3 None admin] PATH_INFO: `/v3/users/99da68de463743fdb9293f16abb81a3b` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:46.173132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6af63b-d22e-4531-a2d5-b6492f9216b3 None admin] RBAC: Authorizing `identity:get_user(user_id=99da68de463743fdb9293f16abb81a3b)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:46.178016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6af63b-d22e-4531-a2d5-b6492f9216b3 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:46.181109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 63/127] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:45 2022] GET /identity/v3/users/99da68de463743fdb9293f16abb81a3b => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:46.183880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6af63b-d22e-4531-a2d5-b6492f9216b3 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:46.232906 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:46.327554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca8c51-8634-42ec-8070-8915e4cb9feb None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:46.368245 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:46.391708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6af63b-d22e-4531-a2d5-b6492f9216b3 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:46.395747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea71f1-a17a-42a5-a3a6-ec9ed59a3bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:46.396975 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea71f1-a17a-42a5-a3a6-ec9ed59a3bb9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:46.397262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea71f1-a17a-42a5-a3a6-ec9ed59a3bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:46.397540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea71f1-a17a-42a5-a3a6-ec9ed59a3bb9 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:46.398227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea71f1-a17a-42a5-a3a6-ec9ed59a3bb9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:46.398538 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea71f1-a17a-42a5-a3a6-ec9ed59a3bb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:46.399984 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea71f1-a17a-42a5-a3a6-ec9ed59a3bb9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:46.417805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 65/128] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:30:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:46.418674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea71f1-a17a-42a5-a3a6-ec9ed59a3bb9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:46.449178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e8277-9f54-4646-a2a5-d85fe423969a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:46.449890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e8277-9f54-4646-a2a5-d85fe423969a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:46.450178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e8277-9f54-4646-a2a5-d85fe423969a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:46.450463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e8277-9f54-4646-a2a5-d85fe423969a None admin] PATH_INFO: `/v3/users/3fea9e7c5b754acaa0233c15976ac3b3` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:46.451641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e8277-9f54-4646-a2a5-d85fe423969a None admin] RBAC: Authorizing `identity:get_user(user_id=3fea9e7c5b754acaa0233c15976ac3b3)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:46.453140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e8277-9f54-4646-a2a5-d85fe423969a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:46.455014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 64/129] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:46 2022] GET /identity/v3/users/3fea9e7c5b754acaa0233c15976ac3b3 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:46.455672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e8277-9f54-4646-a2a5-d85fe423969a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:46.586116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:46.607200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e8277-9f54-4646-a2a5-d85fe423969a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:46.649279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea71f1-a17a-42a5-a3a6-ec9ed59a3bb9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:46.679068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f630b-0889-41f0-a64e-d978f7f986bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:46.679714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f630b-0889-41f0-a64e-d978f7f986bf None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:46.680028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f630b-0889-41f0-a64e-d978f7f986bf None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:46.680313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f630b-0889-41f0-a64e-d978f7f986bf None admin] PATH_INFO: `/v3/users/6fde9f2e823947639d9cd093fe859f57` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:46.704587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f630b-0889-41f0-a64e-d978f7f986bf None admin] RBAC: Authorizing `identity:get_user(user_id=6fde9f2e823947639d9cd093fe859f57)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:46.706103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f630b-0889-41f0-a64e-d978f7f986bf None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:46.707831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 65/130] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:46 2022] GET /identity/v3/users/6fde9f2e823947639d9cd093fe859f57 => generated 289 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:46.708212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e64172-ae0a-4080-8d59-2013a26c533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:46.709214 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f630b-0889-41f0-a64e-d978f7f986bf None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:46.709429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64172-ae0a-4080-8d59-2013a26c533e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:46.709717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64172-ae0a-4080-8d59-2013a26c533e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:46.709995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64172-ae0a-4080-8d59-2013a26c533e None admin] PATH_INFO: `/v3/projects/5f8cbb25889d4713a0d84901578c0cf8` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:46.711262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e64172-ae0a-4080-8d59-2013a26c533e None admin] RBAC: Authorizing `identity:get_project(project_id=5f8cbb25889d4713a0d84901578c0cf8)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:46.712748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e64172-ae0a-4080-8d59-2013a26c533e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:46.714402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 66/131] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:46 2022] GET /identity/v3/projects/5f8cbb25889d4713a0d84901578c0cf8 => generated 312 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:46.719829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e64172-ae0a-4080-8d59-2013a26c533e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:46.787507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:46.859693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f630b-0889-41f0-a64e-d978f7f986bf None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:46.884957 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e64172-ae0a-4080-8d59-2013a26c533e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:46.929357 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417d4da-babb-43cf-95f9-e6c54e2b25c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:46.930004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417d4da-babb-43cf-95f9-e6c54e2b25c9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:46.930295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417d4da-babb-43cf-95f9-e6c54e2b25c9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:46.930585 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417d4da-babb-43cf-95f9-e6c54e2b25c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:46.940590 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38628a-8d0f-48ac-8664-bd04110b1ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:46.941622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38628a-8d0f-48ac-8664-bd04110b1ad2 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:46.942145 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38628a-8d0f-48ac-8664-bd04110b1ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:46.942435 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38628a-8d0f-48ac-8664-bd04110b1ad2 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:46.948670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38628a-8d0f-48ac-8664-bd04110b1ad2 None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:46.951420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38628a-8d0f-48ac-8664-bd04110b1ad2 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:46.954672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 66/132] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:46 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:46.956659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38628a-8d0f-48ac-8664-bd04110b1ad2 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:46.993351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:47.073142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417d4da-babb-43cf-95f9-e6c54e2b25c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:47.075049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417d4da-babb-43cf-95f9-e6c54e2b25c9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:47.164190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38628a-8d0f-48ac-8664-bd04110b1ad2 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:47.176938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:47.202802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b802039-70f7-49a4-8cd4-2599a81e3342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:47.203825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b802039-70f7-49a4-8cd4-2599a81e3342 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:47.204379 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b802039-70f7-49a4-8cd4-2599a81e3342 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:47.204930 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b802039-70f7-49a4-8cd4-2599a81e3342 None admin] PATH_INFO: `/v3/projects/5f8cbb25889d4713a0d84901578c0cf8/users/99da68de463743fdb9293f16abb81a3b/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:47.207220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b802039-70f7-49a4-8cd4-2599a81e3342 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f8cbb25889d4713a0d84901578c0cf8, user_id=99da68de463743fdb9293f16abb81a3b, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:47.208973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b802039-70f7-49a4-8cd4-2599a81e3342 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:47.230586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 67/133] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:47.231441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417d4da-babb-43cf-95f9-e6c54e2b25c9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:47.310312 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 67/134] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:30:46 2022] PUT /identity/v3/projects/5f8cbb25889d4713a0d84901578c0cf8/users/99da68de463743fdb9293f16abb81a3b/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:30:47.317347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a29a8-3d7e-46db-b70e-b6d42eedebe4 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:47.317628 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a29a8-3d7e-46db-b70e-b6d42eedebe4 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:47.317988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a29a8-3d7e-46db-b70e-b6d42eedebe4 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:47.319489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 68/135] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:47.320247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a29a8-3d7e-46db-b70e-b6d42eedebe4 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:47.354772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417d4da-babb-43cf-95f9-e6c54e2b25c9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:47.399988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4010f-524d-4ff1-a358-7846c1248229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:47.402620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4010f-524d-4ff1-a358-7846c1248229 None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:47.402908 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4010f-524d-4ff1-a358-7846c1248229 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:47.403184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4010f-524d-4ff1-a358-7846c1248229 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd/users/3fea9e7c5b754acaa0233c15976ac3b3/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:47.410327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4010f-524d-4ff1-a358-7846c1248229 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1c8960e806347658b410f7278fe7ffd, user_id=3fea9e7c5b754acaa0233c15976ac3b3, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:47.412137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4010f-524d-4ff1-a358-7846c1248229 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:47.458564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:47.497541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a29a8-3d7e-46db-b70e-b6d42eedebe4 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:47.537295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 68/136] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:30:47 2022] PUT /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd/users/3fea9e7c5b754acaa0233c15976ac3b3/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:30:47.537600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfed09-ab4d-4797-880b-28887d0607cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:47.538275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfed09-ab4d-4797-880b-28887d0607cd None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:47.538619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfed09-ab4d-4797-880b-28887d0607cd None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:47.538744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1970f95-da37-457f-9a1d-38de219e2f13 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:47.538900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfed09-ab4d-4797-880b-28887d0607cd None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:47.539770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1970f95-da37-457f-9a1d-38de219e2f13 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:47.540094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1970f95-da37-457f-9a1d-38de219e2f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:47.540802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfed09-ab4d-4797-880b-28887d0607cd None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:47.542563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfed09-ab4d-4797-880b-28887d0607cd None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:47.544618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 69/137] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:47 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:47.550636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfed09-ab4d-4797-880b-28887d0607cd None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:47.562870 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b1970f95-da37-457f-9a1d-38de219e2f13 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:47.564136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b1970f95-da37-457f-9a1d-38de219e2f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:47.569478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:47.702189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfed09-ab4d-4797-880b-28887d0607cd None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:47.769393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfbad4-7de8-42a0-9de4-04486c4c9e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:47.769934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfbad4-7de8-42a0-9de4-04486c4c9e4f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:47.770218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfbad4-7de8-42a0-9de4-04486c4c9e4f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:47.770493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfbad4-7de8-42a0-9de4-04486c4c9e4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:47.771798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfbad4-7de8-42a0-9de4-04486c4c9e4f None admin] RBAC: Adding query filter params (role.id=1c9432f222d8471bb13d0479297cf5d5, scope.project.id=f1c8960e806347658b410f7278fe7ffd, user.id=6fde9f2e823947639d9cd093fe859f57) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:47.772142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfbad4-7de8-42a0-9de4-04486c4c9e4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:47.773235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 69/138] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:47.773939 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfbad4-7de8-42a0-9de4-04486c4c9e4f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:47.780229 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 70/139] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:47 2022] GET /identity/v3/role_assignments?user.id=6fde9f2e823947639d9cd093fe859f57&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=1c9432f222d8471bb13d0479297cf5d5 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:47.991699 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e419af1-ebf3-4b22-b586-8325a33ba8f6 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:47.992096 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e419af1-ebf3-4b22-b586-8325a33ba8f6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:47.992308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e419af1-ebf3-4b22-b586-8325a33ba8f6 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:47.993671 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 70/140] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:47.999231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c2c16-b6a2-4c96-969b-a0d678834bfa None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:47.999524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c2c16-b6a2-4c96-969b-a0d678834bfa None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:47.999811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c2c16-b6a2-4c96-969b-a0d678834bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:48.013108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:48.031362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-750c2c16-b6a2-4c96-969b-a0d678834bfa None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:48.031362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-750c2c16-b6a2-4c96-969b-a0d678834bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:48.112125 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105502c4-f65d-4b8a-ab08-cf9990e53c54 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:48.112442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105502c4-f65d-4b8a-ab08-cf9990e53c54 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:48.112689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105502c4-f65d-4b8a-ab08-cf9990e53c54 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:48.114154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 71/141] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:30:48.120416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2d81b-ffba-439c-9777-733842879632 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:48.120716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2d81b-ffba-439c-9777-733842879632 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:48.120992 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2d81b-ffba-439c-9777-733842879632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:48.126575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 71/142] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:30:48.141688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e5d2d81b-ffba-439c-9777-733842879632 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:48.142898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e5d2d81b-ffba-439c-9777-733842879632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:48.178610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c2c16-b6a2-4c96-969b-a0d678834bfa None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:48.303627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 72/143] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:30:48.372107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c2c16-b6a2-4c96-969b-a0d678834bfa None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:48.420741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850990f6-8300-4594-9f74-81343fb5beec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:48.421386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850990f6-8300-4594-9f74-81343fb5beec None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:48.425967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850990f6-8300-4594-9f74-81343fb5beec None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:48.425967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850990f6-8300-4594-9f74-81343fb5beec None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:48.425967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:48.440772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850990f6-8300-4594-9f74-81343fb5beec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-850990f6-8300-4594-9f74-81343fb5beec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:48.445635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:48.456426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:48.458398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 72/144] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:30:48.460082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2d81b-ffba-439c-9777-733842879632 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:48.560102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:48.599739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2d81b-ffba-439c-9777-733842879632 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:48.623286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850990f6-8300-4594-9f74-81343fb5beec None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:48.648760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014891a-6496-43a1-8326-bff700972120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:48.649401 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014891a-6496-43a1-8326-bff700972120 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:48.649742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014891a-6496-43a1-8326-bff700972120 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:48.650024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014891a-6496-43a1-8326-bff700972120 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:48.650715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014891a-6496-43a1-8326-bff700972120 None admin] RBAC: Adding query filter params (name=member) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:48.651073 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014891a-6496-43a1-8326-bff700972120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:48.652539 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014891a-6496-43a1-8326-bff700972120 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:48.665986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 73/145] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:30:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:48.671550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014891a-6496-43a1-8326-bff700972120 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:48.820447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850990f6-8300-4594-9f74-81343fb5beec None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:48.860336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e524def-a438-44e7-9a36-4770152f0f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:48.860974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e524def-a438-44e7-9a36-4770152f0f3b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:48.861253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e524def-a438-44e7-9a36-4770152f0f3b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:48.861529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e524def-a438-44e7-9a36-4770152f0f3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:48.863650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e524def-a438-44e7-9a36-4770152f0f3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:48.865683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e524def-a438-44e7-9a36-4770152f0f3b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:48.867434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 73/146] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:48.868053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e524def-a438-44e7-9a36-4770152f0f3b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:48.886642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014891a-6496-43a1-8326-bff700972120 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:48.936236 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:48.947746 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb692b2-daa1-4960-a9b0-16899538b9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:48.948430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb692b2-daa1-4960-a9b0-16899538b9cf None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:48.948718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb692b2-daa1-4960-a9b0-16899538b9cf None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:48.950035 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb692b2-daa1-4960-a9b0-16899538b9cf None admin] PATH_INFO: `/v3/users/99da68de463743fdb9293f16abb81a3b` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:48.957233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb692b2-daa1-4960-a9b0-16899538b9cf None admin] RBAC: Authorizing `identity:get_user(user_id=99da68de463743fdb9293f16abb81a3b)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:48.958086 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb692b2-daa1-4960-a9b0-16899538b9cf None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:48.960427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 74/147] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:48 2022] GET /identity/v3/users/99da68de463743fdb9293f16abb81a3b => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:48.964016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb692b2-daa1-4960-a9b0-16899538b9cf None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:49.013641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:49.061908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e524def-a438-44e7-9a36-4770152f0f3b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:49.104078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb303092-4817-40fc-b042-5e06257357e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:49.105418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb303092-4817-40fc-b042-5e06257357e8 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:49.105925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb303092-4817-40fc-b042-5e06257357e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:49.106400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb303092-4817-40fc-b042-5e06257357e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:49.119523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb303092-4817-40fc-b042-5e06257357e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-cb303092-4817-40fc-b042-5e06257357e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:49.126876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:49.127827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:30:49.128646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:49.131498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 74/148] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:30:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:49.132653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb303092-4817-40fc-b042-5e06257357e8 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:49.200569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb692b2-daa1-4960-a9b0-16899538b9cf None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:49.258272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb303092-4817-40fc-b042-5e06257357e8 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:49.287326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:49.301270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb142c8-27ed-4f54-a4f8-2baa9bf2f55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:49.301870 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb142c8-27ed-4f54-a4f8-2baa9bf2f55b None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:49.302161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb142c8-27ed-4f54-a4f8-2baa9bf2f55b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:49.302491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb142c8-27ed-4f54-a4f8-2baa9bf2f55b None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:49.303138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb142c8-27ed-4f54-a4f8-2baa9bf2f55b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:49.303950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd7a6e-21e8-4cd1-b1e6-26b6b9623a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:49.304627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb142c8-27ed-4f54-a4f8-2baa9bf2f55b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:49.305357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd7a6e-21e8-4cd1-b1e6-26b6b9623a98 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:49.305839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd7a6e-21e8-4cd1-b1e6-26b6b9623a98 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:49.306328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd7a6e-21e8-4cd1-b1e6-26b6b9623a98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:49.390587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd7a6e-21e8-4cd1-b1e6-26b6b9623a98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:49.392515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd7a6e-21e8-4cd1-b1e6-26b6b9623a98 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:49.470408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 75/149] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:30:48 2022] POST /identity/v3/users => generated 283 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:30:49.471491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb142c8-27ed-4f54-a4f8-2baa9bf2f55b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:49.514214 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 75/150] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:49.522183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd7a6e-21e8-4cd1-b1e6-26b6b9623a98 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:49.555196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:49.655377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd7a6e-21e8-4cd1-b1e6-26b6b9623a98 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:49.659043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:49.702349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25723ea3-7b57-46fe-bdaf-c4973cf67e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:49.703009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25723ea3-7b57-46fe-bdaf-c4973cf67e7e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:49.703299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25723ea3-7b57-46fe-bdaf-c4973cf67e7e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:49.703584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25723ea3-7b57-46fe-bdaf-c4973cf67e7e None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:49.705155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25723ea3-7b57-46fe-bdaf-c4973cf67e7e None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:49.706887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25723ea3-7b57-46fe-bdaf-c4973cf67e7e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:49.710565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 76/151] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:49 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:49.718844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25723ea3-7b57-46fe-bdaf-c4973cf67e7e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:49.753449 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb142c8-27ed-4f54-a4f8-2baa9bf2f55b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:49.829337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de248943-d42b-4a64-ba42-2b11c183ac66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:49.830243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de248943-d42b-4a64-ba42-2b11c183ac66 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:49.830675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de248943-d42b-4a64-ba42-2b11c183ac66 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:49.831097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de248943-d42b-4a64-ba42-2b11c183ac66 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:49.831962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de248943-d42b-4a64-ba42-2b11c183ac66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:49.832419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de248943-d42b-4a64-ba42-2b11c183ac66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:49.834548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de248943-d42b-4a64-ba42-2b11c183ac66 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:49.850213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 76/152] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:30:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:49.857131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de248943-d42b-4a64-ba42-2b11c183ac66 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:49.911065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:49.934896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25723ea3-7b57-46fe-bdaf-c4973cf67e7e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:49.992461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c1c23-d213-4d3d-80c5-ce32a4f7a92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:49.993602 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c1c23-d213-4d3d-80c5-ce32a4f7a92e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:49.993892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c1c23-d213-4d3d-80c5-ce32a4f7a92e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:49.994168 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c1c23-d213-4d3d-80c5-ce32a4f7a92e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:49.995072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c1c23-d213-4d3d-80c5-ce32a4f7a92e None admin] RBAC: Adding query filter params (role.id=0166494127b14615900b2c121908b2ba, scope.project.id=f1c8960e806347658b410f7278fe7ffd, user.id=99da68de463743fdb9293f16abb81a3b) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:49.995526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c1c23-d213-4d3d-80c5-ce32a4f7a92e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:49.997397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c1c23-d213-4d3d-80c5-ce32a4f7a92e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:50.006221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 77/153] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:49 2022] GET /identity/v3/role_assignments?user.id=99da68de463743fdb9293f16abb81a3b&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=0166494127b14615900b2c121908b2ba => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:50.007810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf98a-3aba-4d92-be90-16e29d8f362f None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:50.008354 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf98a-3aba-4d92-be90-16e29d8f362f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:50.008859 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf98a-3aba-4d92-be90-16e29d8f362f None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:50.010767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 78/154] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:50.016641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c2f5f-84fe-4855-83a9-6892461f0817 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:50.017410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c2f5f-84fe-4855-83a9-6892461f0817 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:50.017906 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c2f5f-84fe-4855-83a9-6892461f0817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:50.043046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-543c2f5f-84fe-4855-83a9-6892461f0817 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:50.045369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-543c2f5f-84fe-4855-83a9-6892461f0817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:50.062569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de248943-d42b-4a64-ba42-2b11c183ac66 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:50.099078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:50.142197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d240ce2-9b35-4db7-9acc-31ed34bd0c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:50.142856 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d240ce2-9b35-4db7-9acc-31ed34bd0c9d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:50.143143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d240ce2-9b35-4db7-9acc-31ed34bd0c9d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:50.143424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d240ce2-9b35-4db7-9acc-31ed34bd0c9d None admin] PATH_INFO: `/v3/users/99da68de463743fdb9293f16abb81a3b` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:50.144994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d240ce2-9b35-4db7-9acc-31ed34bd0c9d None admin] RBAC: Authorizing `identity:get_user(user_id=99da68de463743fdb9293f16abb81a3b)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:50.149106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d240ce2-9b35-4db7-9acc-31ed34bd0c9d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:50.151174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 77/155] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:49 2022] GET /identity/v3/users/99da68de463743fdb9293f16abb81a3b => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:50.157655 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d240ce2-9b35-4db7-9acc-31ed34bd0c9d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:50.196213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 79/156] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:50.317627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d240ce2-9b35-4db7-9acc-31ed34bd0c9d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:50.336240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab58e94-d90c-45df-b07e-eaf6325b7e2a None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:50.336787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab58e94-d90c-45df-b07e-eaf6325b7e2a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:50.337021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab58e94-d90c-45df-b07e-eaf6325b7e2a None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:50.338493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 80/157] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:30:50.345700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525b5b5-6f19-4ca8-80ae-2aaa16f9d045 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:50.346120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525b5b5-6f19-4ca8-80ae-2aaa16f9d045 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:50.346503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525b5b5-6f19-4ca8-80ae-2aaa16f9d045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:50.368059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d97fac-9fb7-450c-ae5c-775adf2eda41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:50.368059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d97fac-9fb7-450c-ae5c-775adf2eda41 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:50.368059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d97fac-9fb7-450c-ae5c-775adf2eda41 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:50.368059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d97fac-9fb7-450c-ae5c-775adf2eda41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:50.372908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-d525b5b5-6f19-4ca8-80ae-2aaa16f9d045 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:50.374403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-d525b5b5-6f19-4ca8-80ae-2aaa16f9d045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:50.458176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:50.482970 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d97fac-9fb7-450c-ae5c-775adf2eda41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:50.493730 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d97fac-9fb7-450c-ae5c-775adf2eda41 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:50.497110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 81/158] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:30:50.500328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eaba21-9f80-4e36-8757-f2e74a43984b None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:50.500621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eaba21-9f80-4e36-8757-f2e74a43984b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:50.500900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eaba21-9f80-4e36-8757-f2e74a43984b None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:50.502291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 82/159] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:50.509129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47916-710e-4fe5-9b66-8fa2982999ec None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:50.509427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47916-710e-4fe5-9b66-8fa2982999ec None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:50.509704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47916-710e-4fe5-9b66-8fa2982999ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:50.533644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-02c47916-710e-4fe5-9b66-8fa2982999ec None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:50.535162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-02c47916-710e-4fe5-9b66-8fa2982999ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:50.550240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:50.628572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 83/160] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:30:50.652055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 78/161] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:50.661023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c47916-710e-4fe5-9b66-8fa2982999ec None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:50.675362 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d97fac-9fb7-450c-ae5c-775adf2eda41 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:50.798677 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:50.830854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d97fac-9fb7-450c-ae5c-775adf2eda41 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:50.872504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c47916-710e-4fe5-9b66-8fa2982999ec None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:50.908146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd657e0-7e85-4cf4-8631-8dfca7e41599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:50.909063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd657e0-7e85-4cf4-8631-8dfca7e41599 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:50.909531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd657e0-7e85-4cf4-8631-8dfca7e41599 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:50.909967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd657e0-7e85-4cf4-8631-8dfca7e41599 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:50.913502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a79a5b-f39c-4775-ba45-aac3ed7b567d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:50.914413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a79a5b-f39c-4775-ba45-aac3ed7b567d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:50.914990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a79a5b-f39c-4775-ba45-aac3ed7b567d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:50.915454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a79a5b-f39c-4775-ba45-aac3ed7b567d None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:50.916868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a79a5b-f39c-4775-ba45-aac3ed7b567d None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:50.918540 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a79a5b-f39c-4775-ba45-aac3ed7b567d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:50.920445 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 84/162] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:50 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:50.921417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a79a5b-f39c-4775-ba45-aac3ed7b567d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:50.922199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd657e0-7e85-4cf4-8631-8dfca7e41599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-8bd657e0-7e85-4cf4-8631-8dfca7e41599 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:50.928165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:50.929072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:30:50.929869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 79/163] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:50.930522 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd657e0-7e85-4cf4-8631-8dfca7e41599 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:50.964259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:51.044221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a79a5b-f39c-4775-ba45-aac3ed7b567d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:51.070118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd657e0-7e85-4cf4-8631-8dfca7e41599 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:51.092880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e009c2b-22ca-4007-8466-2f4c36684631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:51.093801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e009c2b-22ca-4007-8466-2f4c36684631 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:51.094087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e009c2b-22ca-4007-8466-2f4c36684631 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:51.094362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e009c2b-22ca-4007-8466-2f4c36684631 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:51.110227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e009c2b-22ca-4007-8466-2f4c36684631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-6e009c2b-22ca-4007-8466-2f4c36684631 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:51.115682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:51.116594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 85/164] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:51.117761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e009c2b-22ca-4007-8466-2f4c36684631 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:51.126335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e3e7a-76a5-41a5-9df4-8631582ba4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:51.126965 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e3e7a-76a5-41a5-9df4-8631582ba4ce None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:51.127252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e3e7a-76a5-41a5-9df4-8631582ba4ce None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:51.127568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e3e7a-76a5-41a5-9df4-8631582ba4ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:51.128300 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e3e7a-76a5-41a5-9df4-8631582ba4ce None admin] RBAC: Adding query filter params (role.id=e0822035b245457eade1325bbe888913, scope.project.id=f1c8960e806347658b410f7278fe7ffd, user.id=99da68de463743fdb9293f16abb81a3b) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:51.128615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e3e7a-76a5-41a5-9df4-8631582ba4ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:51.130174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e3e7a-76a5-41a5-9df4-8631582ba4ce None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:51.137354 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 80/165] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:50 2022] GET /identity/v3/role_assignments?user.id=99da68de463743fdb9293f16abb81a3b&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=e0822035b245457eade1325bbe888913 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:51.138570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e3e7a-76a5-41a5-9df4-8631582ba4ce None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:51.184256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:51.211980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:51.278950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e009c2b-22ca-4007-8466-2f4c36684631 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:51.317544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e3e7a-76a5-41a5-9df4-8631582ba4ce None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:51.352852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b66d0-aece-4bcb-8a0c-4d71f55c1e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:51.354155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b66d0-aece-4bcb-8a0c-4d71f55c1e89 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:51.354642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b66d0-aece-4bcb-8a0c-4d71f55c1e89 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:51.355119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b66d0-aece-4bcb-8a0c-4d71f55c1e89 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:51.357009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b66d0-aece-4bcb-8a0c-4d71f55c1e89 None admin] RBAC: Adding query filter params (name=member) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:51.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b66d0-aece-4bcb-8a0c-4d71f55c1e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:51.361666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b66d0-aece-4bcb-8a0c-4d71f55c1e89 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:51.375472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 86/166] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:30:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:51.377821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b66d0-aece-4bcb-8a0c-4d71f55c1e89 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:51.398907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d435c65-7b69-496f-b65f-614c3b9760f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:51.401492 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d435c65-7b69-496f-b65f-614c3b9760f9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:51.401990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d435c65-7b69-496f-b65f-614c3b9760f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:51.402502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d435c65-7b69-496f-b65f-614c3b9760f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:51.419940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d435c65-7b69-496f-b65f-614c3b9760f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-7d435c65-7b69-496f-b65f-614c3b9760f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:51.428434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:51.432802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 81/167] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:30:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:51.433690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d435c65-7b69-496f-b65f-614c3b9760f9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:51.571393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:51.603855 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b66d0-aece-4bcb-8a0c-4d71f55c1e89 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:51.638181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d435c65-7b69-496f-b65f-614c3b9760f9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:51.666483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62271-4da6-42dd-8a40-af11c70b4974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:51.667943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62271-4da6-42dd-8a40-af11c70b4974 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:51.668225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62271-4da6-42dd-8a40-af11c70b4974 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:51.668500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62271-4da6-42dd-8a40-af11c70b4974 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:51.669177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab62271-4da6-42dd-8a40-af11c70b4974 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:51.669479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab62271-4da6-42dd-8a40-af11c70b4974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:51.670869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab62271-4da6-42dd-8a40-af11c70b4974 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:51.673912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:51.690437 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 87/168] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:30:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:51.691111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62271-4da6-42dd-8a40-af11c70b4974 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:51.710961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7358d75-c62e-4a1d-bdf2-c9a8701938c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:51.711645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7358d75-c62e-4a1d-bdf2-c9a8701938c9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:51.711941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7358d75-c62e-4a1d-bdf2-c9a8701938c9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:51.712270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7358d75-c62e-4a1d-bdf2-c9a8701938c9 None admin] PATH_INFO: `/v3/users/99da68de463743fdb9293f16abb81a3b` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:51.713894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7358d75-c62e-4a1d-bdf2-c9a8701938c9 None admin] RBAC: Authorizing `identity:get_user(user_id=99da68de463743fdb9293f16abb81a3b)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:51.716930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7358d75-c62e-4a1d-bdf2-c9a8701938c9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:51.721132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 82/169] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:51 2022] GET /identity/v3/users/99da68de463743fdb9293f16abb81a3b => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:51.722152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7358d75-c62e-4a1d-bdf2-c9a8701938c9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:51.903717 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62271-4da6-42dd-8a40-af11c70b4974 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:51.908011 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7358d75-c62e-4a1d-bdf2-c9a8701938c9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:51.948559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399e743-1d21-4a49-8703-94a89c1cf19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:51.949202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399e743-1d21-4a49-8703-94a89c1cf19e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:51.949479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399e743-1d21-4a49-8703-94a89c1cf19e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:51.949756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399e743-1d21-4a49-8703-94a89c1cf19e None admin] PATH_INFO: `/v3/users/6fde9f2e823947639d9cd093fe859f57` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:51.950910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399e743-1d21-4a49-8703-94a89c1cf19e None admin] RBAC: Authorizing `identity:get_user(user_id=6fde9f2e823947639d9cd093fe859f57)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:51.956403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399e743-1d21-4a49-8703-94a89c1cf19e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:51.958189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 83/170] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:51 2022] GET /identity/v3/users/6fde9f2e823947639d9cd093fe859f57 => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:51.959338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399e743-1d21-4a49-8703-94a89c1cf19e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:51.961128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fcb1f-e619-4e53-9260-12ff38465c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:51.961736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fcb1f-e619-4e53-9260-12ff38465c67 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:51.962015 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fcb1f-e619-4e53-9260-12ff38465c67 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:51.962338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fcb1f-e619-4e53-9260-12ff38465c67 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:51.963014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fcb1f-e619-4e53-9260-12ff38465c67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:51.963316 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fcb1f-e619-4e53-9260-12ff38465c67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:51.964718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fcb1f-e619-4e53-9260-12ff38465c67 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:51.979497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:51.983411 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 88/171] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:51.989418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fcb1f-e619-4e53-9260-12ff38465c67 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:52.095784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:52.166590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399e743-1d21-4a49-8703-94a89c1cf19e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:52.178958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fcb1f-e619-4e53-9260-12ff38465c67 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:52.217340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7188b-d5a4-4458-b703-1a7301c51bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:52.217972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7188b-d5a4-4458-b703-1a7301c51bc4 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:52.218302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7188b-d5a4-4458-b703-1a7301c51bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:52.218582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7188b-d5a4-4458-b703-1a7301c51bc4 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:52.219759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7188b-d5a4-4458-b703-1a7301c51bc4 None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:52.221370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7188b-d5a4-4458-b703-1a7301c51bc4 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:52.223036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 89/172] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:51 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:52.223984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7188b-d5a4-4458-b703-1a7301c51bc4 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:52.250031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd2df6-28e3-48bb-8fe3-d13626522fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:52.250668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd2df6-28e3-48bb-8fe3-d13626522fb0 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:52.250954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd2df6-28e3-48bb-8fe3-d13626522fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:52.251232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd2df6-28e3-48bb-8fe3-d13626522fb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:52.380087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7188b-d5a4-4458-b703-1a7301c51bc4 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:52.392532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd2df6-28e3-48bb-8fe3-d13626522fb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:52.394312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd2df6-28e3-48bb-8fe3-d13626522fb0 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:52.410900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:52.415894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:52.435792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498484e-8411-4afa-aef6-62290819017a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:52.435792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498484e-8411-4afa-aef6-62290819017a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:52.435792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498484e-8411-4afa-aef6-62290819017a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:52.435792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498484e-8411-4afa-aef6-62290819017a None admin] PATH_INFO: `/v3/users/3fea9e7c5b754acaa0233c15976ac3b3` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:52.436275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498484e-8411-4afa-aef6-62290819017a None admin] RBAC: Authorizing `identity:get_user(user_id=3fea9e7c5b754acaa0233c15976ac3b3)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:52.436275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498484e-8411-4afa-aef6-62290819017a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:52.436275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 90/173] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:52 2022] GET /identity/v3/users/3fea9e7c5b754acaa0233c15976ac3b3 => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:52.436275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498484e-8411-4afa-aef6-62290819017a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:52.555546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498484e-8411-4afa-aef6-62290819017a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:52.593911 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eb0bb-e428-4356-8f72-2471edb11e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:52.594590 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eb0bb-e428-4356-8f72-2471edb11e4e None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:52.594878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eb0bb-e428-4356-8f72-2471edb11e4e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:52.595158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eb0bb-e428-4356-8f72-2471edb11e4e None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd/users/6fde9f2e823947639d9cd093fe859f57/roles/1c9432f222d8471bb13d0479297cf5d5` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:52.597372 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eb0bb-e428-4356-8f72-2471edb11e4e None admin] RBAC: Authorizing `identity:create_grant(project_id=f1c8960e806347658b410f7278fe7ffd, user_id=6fde9f2e823947639d9cd093fe859f57, role_id=1c9432f222d8471bb13d0479297cf5d5)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:52.598956 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eb0bb-e428-4356-8f72-2471edb11e4e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:52.645189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 84/174] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:52.646073 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd2df6-28e3-48bb-8fe3-d13626522fb0 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:52.669077 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 91/175] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:30:52 2022] PUT /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd/users/6fde9f2e823947639d9cd093fe859f57/roles/1c9432f222d8471bb13d0479297cf5d5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:30:52.669914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eb0bb-e428-4356-8f72-2471edb11e4e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:52.772359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:52.808194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd2df6-28e3-48bb-8fe3-d13626522fb0 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:52.816646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:52.846809 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c119f-ef16-44c2-90d5-cb1645d3f637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:52.847440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c119f-ef16-44c2-90d5-cb1645d3f637 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:52.847723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c119f-ef16-44c2-90d5-cb1645d3f637 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:52.848029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c119f-ef16-44c2-90d5-cb1645d3f637 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:52.850638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eb0bb-e428-4356-8f72-2471edb11e4e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:52.915725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d71ccc-a676-4c2c-abfe-a76a37f45c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:52.916773 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d71ccc-a676-4c2c-abfe-a76a37f45c0e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:52.917272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d71ccc-a676-4c2c-abfe-a76a37f45c0e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:52.917749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d71ccc-a676-4c2c-abfe-a76a37f45c0e None admin] PATH_INFO: `/v3/projects/5f8cbb25889d4713a0d84901578c0cf8` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:52.924727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d71ccc-a676-4c2c-abfe-a76a37f45c0e None admin] RBAC: Authorizing `identity:get_project(project_id=5f8cbb25889d4713a0d84901578c0cf8)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:52.926813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d71ccc-a676-4c2c-abfe-a76a37f45c0e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:52.927119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c119f-ef16-44c2-90d5-cb1645d3f637 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:52.929349 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c119f-ef16-44c2-90d5-cb1645d3f637 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:52.929571 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 92/176] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:52 2022] GET /identity/v3/projects/5f8cbb25889d4713a0d84901578c0cf8 => generated 312 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:52.930962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e30e4-cb41-422d-92fc-67167bdf4ec3 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:52.931356 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e30e4-cb41-422d-92fc-67167bdf4ec3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:52.931739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e30e4-cb41-422d-92fc-67167bdf4ec3 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:52.933191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 93/177] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:52.938725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e103b-255e-4d64-9112-8797333bae3b None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:52.939297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e103b-255e-4d64-9112-8797333bae3b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:52.939785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e103b-255e-4d64-9112-8797333bae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:52.959148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-c60e103b-255e-4d64-9112-8797333bae3b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:52.960717 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c60e103b-255e-4d64-9112-8797333bae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:53.034256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 94/178] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:30:53.337104 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e103b-255e-4d64-9112-8797333bae3b None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:53.337256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 85/179] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:53.337256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c119f-ef16-44c2-90d5-cb1645d3f637 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:53.337256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:53.337473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:53.337473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e103b-255e-4d64-9112-8797333bae3b None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:53.337617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c119f-ef16-44c2-90d5-cb1645d3f637 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:53.337697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ea661-abd5-458b-91b9-ac7c5cd08d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:53.337697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ea661-abd5-458b-91b9-ac7c5cd08d88 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:53.337697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ea661-abd5-458b-91b9-ac7c5cd08d88 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:53.337697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ea661-abd5-458b-91b9-ac7c5cd08d88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:53.337977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ea661-abd5-458b-91b9-ac7c5cd08d88 None admin] RBAC: Adding query filter params (role.id=0166494127b14615900b2c121908b2ba, scope.project.id=5f8cbb25889d4713a0d84901578c0cf8, user.id=99da68de463743fdb9293f16abb81a3b) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:53.337977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ea661-abd5-458b-91b9-ac7c5cd08d88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:53.337977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ea661-abd5-458b-91b9-ac7c5cd08d88 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:53.338177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97558ad9-b24b-4675-b018-3840171f9971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:53.338177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97558ad9-b24b-4675-b018-3840171f9971 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:53.338177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97558ad9-b24b-4675-b018-3840171f9971 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:53.338177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97558ad9-b24b-4675-b018-3840171f9971 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:53.338442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 95/180] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:53 2022] GET /identity/v3/role_assignments?user.id=99da68de463743fdb9293f16abb81a3b&scope.project.id=5f8cbb25889d4713a0d84901578c0cf8&role.id=0166494127b14615900b2c121908b2ba => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:53.338515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97558ad9-b24b-4675-b018-3840171f9971 None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:53.338588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fddfa-232f-4ce5-b983-ee3fd5af8c4e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:53.338588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fddfa-232f-4ce5-b983-ee3fd5af8c4e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:53.338588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fddfa-232f-4ce5-b983-ee3fd5af8c4e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:53.338779 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97558ad9-b24b-4675-b018-3840171f9971 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:53.338850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 96/181] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:53.338922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 86/182] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:53 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:53.338996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855d1c-ed8f-45d6-8ba9-9a2920061c55 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:53.338996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855d1c-ed8f-45d6-8ba9-9a2920061c55 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:53.338996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855d1c-ed8f-45d6-8ba9-9a2920061c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:53.339187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97558ad9-b24b-4675-b018-3840171f9971 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:53.339264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-73855d1c-ed8f-45d6-8ba9-9a2920061c55 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:53.339264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-73855d1c-ed8f-45d6-8ba9-9a2920061c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:53.339264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 97/183] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:53.339464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97558ad9-b24b-4675-b018-3840171f9971 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:53.339464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:53.363225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f50881-89b0-4f8c-a18a-0413640f0b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:53.363853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f50881-89b0-4f8c-a18a-0413640f0b67 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:53.364158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f50881-89b0-4f8c-a18a-0413640f0b67 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:53.364432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f50881-89b0-4f8c-a18a-0413640f0b67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:53.365824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f50881-89b0-4f8c-a18a-0413640f0b67 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=f1c8960e806347658b410f7278fe7ffd, user.id=3fea9e7c5b754acaa0233c15976ac3b3) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:53.366133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f50881-89b0-4f8c-a18a-0413640f0b67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:53.367699 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f50881-89b0-4f8c-a18a-0413640f0b67 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:53.374665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 87/184] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:53 2022] GET /identity/v3/role_assignments?user.id=3fea9e7c5b754acaa0233c15976ac3b3&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=6ba5a5add2174983be37258aad1ee404 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:53.520508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43a088-5eea-472a-83a8-efc2427486dd None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:53.522149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43a088-5eea-472a-83a8-efc2427486dd None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:53.522645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43a088-5eea-472a-83a8-efc2427486dd None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:53.524701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 98/185] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:53.530146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fc02d-9651-4a09-9695-3ffcfebb728d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:53.530639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fc02d-9651-4a09-9695-3ffcfebb728d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:53.530721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fc02d-9651-4a09-9695-3ffcfebb728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:53.587889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43a088-5eea-472a-83a8-efc2427486dd None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:53.647729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7a1fc02d-9651-4a09-9695-3ffcfebb728d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:53.649331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7a1fc02d-9651-4a09-9695-3ffcfebb728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:53.681537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:53.742390 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43a088-5eea-472a-83a8-efc2427486dd None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:53.744069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 88/186] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:53.745715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fc02d-9651-4a09-9695-3ffcfebb728d None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:53.796083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2027ca-b2ee-4d56-aa1d-6afca0a3380d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:53.797291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2027ca-b2ee-4d56-aa1d-6afca0a3380d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:53.797802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2027ca-b2ee-4d56-aa1d-6afca0a3380d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:53.798601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2027ca-b2ee-4d56-aa1d-6afca0a3380d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:53.814711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2027ca-b2ee-4d56-aa1d-6afca0a3380d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-af2027ca-b2ee-4d56-aa1d-6afca0a3380d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:53.820846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:53.821870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:53.822701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:53.827081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:53.827225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 99/187] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:53.828688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2027ca-b2ee-4d56-aa1d-6afca0a3380d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:53.877308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fc02d-9651-4a09-9695-3ffcfebb728d None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:53.924229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc2ffd-09bf-4901-8153-0dfb064c2ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:53.926018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc2ffd-09bf-4901-8153-0dfb064c2ee9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:53.926495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc2ffd-09bf-4901-8153-0dfb064c2ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:53.927215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc2ffd-09bf-4901-8153-0dfb064c2ee9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:53.938923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2027ca-b2ee-4d56-aa1d-6afca0a3380d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:53.943069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc2ffd-09bf-4901-8153-0dfb064c2ee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:53.948020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-85bc2ffd-09bf-4901-8153-0dfb064c2ee9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:53.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:53.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:30:53.951628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 89/188] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:30:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:53.959993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc2ffd-09bf-4901-8153-0dfb064c2ee9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:53.987109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00a1b3-a615-40cb-b320-b8c4845aae41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:53.987726 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00a1b3-a615-40cb-b320-b8c4845aae41 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:53.988045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00a1b3-a615-40cb-b320-b8c4845aae41 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:53.988327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00a1b3-a615-40cb-b320-b8c4845aae41 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:53.989007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00a1b3-a615-40cb-b320-b8c4845aae41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:53.989371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00a1b3-a615-40cb-b320-b8c4845aae41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:53.991477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00a1b3-a615-40cb-b320-b8c4845aae41 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:54.000697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 100/189] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:54.007019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00a1b3-a615-40cb-b320-b8c4845aae41 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:54.074166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc2ffd-09bf-4901-8153-0dfb064c2ee9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:54.103193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:54.107944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00a1b3-a615-40cb-b320-b8c4845aae41 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:54.111353 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1cc6a-9b9a-4c77-8c61-a21cd635b1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:54.112004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1cc6a-9b9a-4c77-8c61-a21cd635b1cd None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:54.118695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1cc6a-9b9a-4c77-8c61-a21cd635b1cd None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:54.118695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1cc6a-9b9a-4c77-8c61-a21cd635b1cd None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:54.118695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1cc6a-9b9a-4c77-8c61-a21cd635b1cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:54.118695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1cc6a-9b9a-4c77-8c61-a21cd635b1cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:54.118695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1cc6a-9b9a-4c77-8c61-a21cd635b1cd None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:54.125960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 90/190] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:30:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:54.130750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1cc6a-9b9a-4c77-8c61-a21cd635b1cd None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:54.147321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cff11-731a-4335-8c7f-551548e221fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:54.147993 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cff11-731a-4335-8c7f-551548e221fc None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:54.148282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cff11-731a-4335-8c7f-551548e221fc None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:54.148565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cff11-731a-4335-8c7f-551548e221fc None admin] PATH_INFO: `/v3/users/929b1a056a8f439387ff4b804edcbb0e` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:54.164262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cff11-731a-4335-8c7f-551548e221fc None admin] RBAC: Authorizing `identity:get_user(user_id=929b1a056a8f439387ff4b804edcbb0e)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:54.166062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cff11-731a-4335-8c7f-551548e221fc None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:54.167688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 101/191] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:54 2022] GET /identity/v3/users/929b1a056a8f439387ff4b804edcbb0e => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:54.173024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cff11-731a-4335-8c7f-551548e221fc None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:54.203145 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:54.242681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1cc6a-9b9a-4c77-8c61-a21cd635b1cd None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:54.277657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec842f-153d-499f-aff4-ceb5d8de700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:54.278271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec842f-153d-499f-aff4-ceb5d8de700d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:54.278558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec842f-153d-499f-aff4-ceb5d8de700d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:54.278837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec842f-153d-499f-aff4-ceb5d8de700d None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:54.279844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cff11-731a-4335-8c7f-551548e221fc None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:54.279980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec842f-153d-499f-aff4-ceb5d8de700d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:54.281448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec842f-153d-499f-aff4-ceb5d8de700d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:54.283195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 91/192] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:54.283897 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec842f-153d-499f-aff4-ceb5d8de700d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:54.321248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6745a-9336-45c6-b227-ef640b4257d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:54.322229 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6745a-9336-45c6-b227-ef640b4257d7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:54.322582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6745a-9336-45c6-b227-ef640b4257d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:54.322861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6745a-9336-45c6-b227-ef640b4257d7 None admin] PATH_INFO: `/v3/users/929b1a056a8f439387ff4b804edcbb0e` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:54.324169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6745a-9336-45c6-b227-ef640b4257d7 None admin] RBAC: Authorizing `identity:get_user(user_id=929b1a056a8f439387ff4b804edcbb0e)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:54.326006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6745a-9336-45c6-b227-ef640b4257d7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:54.332468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 102/193] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:54 2022] GET /identity/v3/users/929b1a056a8f439387ff4b804edcbb0e => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:54.332468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6745a-9336-45c6-b227-ef640b4257d7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:54.359584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:54.403531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec842f-153d-499f-aff4-ceb5d8de700d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:54.435498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:54.441303 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026d7e2-2355-4648-8582-a562bbe1cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:54.441930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026d7e2-2355-4648-8582-a562bbe1cb8c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:54.442050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6745a-9336-45c6-b227-ef640b4257d7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:54.442315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026d7e2-2355-4648-8582-a562bbe1cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:54.442515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026d7e2-2355-4648-8582-a562bbe1cb8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:54.480763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeb15a-3313-496f-b269-3f8c7d3c0e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:54.481413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb15a-3313-496f-b269-3f8c7d3c0e4a None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:54.481707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb15a-3313-496f-b269-3f8c7d3c0e4a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:54.481992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb15a-3313-496f-b269-3f8c7d3c0e4a None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:54.482656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeb15a-3313-496f-b269-3f8c7d3c0e4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:54.484317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeb15a-3313-496f-b269-3f8c7d3c0e4a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:54.516818 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026d7e2-2355-4648-8582-a562bbe1cb8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:54.518291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026d7e2-2355-4648-8582-a562bbe1cb8c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:54.577836 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:54.610260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 103/194] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:30:54 2022] POST /identity/v3/users => generated 297 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:30:54.610260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeb15a-3313-496f-b269-3f8c7d3c0e4a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:54.648105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 92/195] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:54.654456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026d7e2-2355-4648-8582-a562bbe1cb8c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:54.742312 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeb15a-3313-496f-b269-3f8c7d3c0e4a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:54.783149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a6f96-fecc-43c7-bcda-49bfa7dbece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:54.783777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a6f96-fecc-43c7-bcda-49bfa7dbece3 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:54.784093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a6f96-fecc-43c7-bcda-49bfa7dbece3 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:54.784372 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a6f96-fecc-43c7-bcda-49bfa7dbece3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:54.784474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026d7e2-2355-4648-8582-a562bbe1cb8c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:54.808373 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:54.826490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e56889-2d12-4b88-ae56-f71655a9f4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:54.827313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e56889-2d12-4b88-ae56-f71655a9f4ab None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:54.827604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e56889-2d12-4b88-ae56-f71655a9f4ab None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:54.827936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e56889-2d12-4b88-ae56-f71655a9f4ab None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:54.829637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e56889-2d12-4b88-ae56-f71655a9f4ab None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:54.831152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e56889-2d12-4b88-ae56-f71655a9f4ab None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:54.834460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 93/196] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:54 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:54.840974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e56889-2d12-4b88-ae56-f71655a9f4ab None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:54.863634 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a6f96-fecc-43c7-bcda-49bfa7dbece3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:54.872965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a6f96-fecc-43c7-bcda-49bfa7dbece3 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:54.874048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:54.976259 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e56889-2d12-4b88-ae56-f71655a9f4ab None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:55.007179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 104/197] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:55.012962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a6f96-fecc-43c7-bcda-49bfa7dbece3 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:55.021270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3334ad-6503-4b8a-aa17-6977ed38638f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:55.022463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3334ad-6503-4b8a-aa17-6977ed38638f None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:55.022750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3334ad-6503-4b8a-aa17-6977ed38638f None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:55.023024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3334ad-6503-4b8a-aa17-6977ed38638f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:55.024403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3334ad-6503-4b8a-aa17-6977ed38638f None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=1ee18025f6d34d0daf407c44c5f28280, user.id=929b1a056a8f439387ff4b804edcbb0e) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:55.024728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3334ad-6503-4b8a-aa17-6977ed38638f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:55.026225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3334ad-6503-4b8a-aa17-6977ed38638f None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:55.035813 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 94/198] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:54 2022] GET /identity/v3/role_assignments?user.id=929b1a056a8f439387ff4b804edcbb0e&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=6ba5a5add2174983be37258aad1ee404 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:55.041086 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf99fe-932e-47b8-9139-8d02f1542936 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:55.041396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf99fe-932e-47b8-9139-8d02f1542936 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:55.041581 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf99fe-932e-47b8-9139-8d02f1542936 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:55.043196 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 95/199] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:30:55.050718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6be74f-43b4-4b70-89c0-189a96767dc7 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:55.051051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6be74f-43b4-4b70-89c0-189a96767dc7 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:55.051281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6be74f-43b4-4b70-89c0-189a96767dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:55.073597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ee6be74f-43b4-4b70-89c0-189a96767dc7 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:55.074795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ee6be74f-43b4-4b70-89c0-189a96767dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:55.095836 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:55.168790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a6f96-fecc-43c7-bcda-49bfa7dbece3 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:55.169862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 96/200] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:55.211775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32e297-7ad9-4950-8507-f96391e07352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:55.212471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32e297-7ad9-4950-8507-f96391e07352 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:55.212761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32e297-7ad9-4950-8507-f96391e07352 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:55.213042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32e297-7ad9-4950-8507-f96391e07352 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:55.214244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32e297-7ad9-4950-8507-f96391e07352 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:55.215733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32e297-7ad9-4950-8507-f96391e07352 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:55.217799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 105/201] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:55 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:55.223956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6be74f-43b4-4b70-89c0-189a96767dc7 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:55.252700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:55.334925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6be74f-43b4-4b70-89c0-189a96767dc7 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:55.372208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ae5f4-9c21-4b76-ab58-7a5854345180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:55.372895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ae5f4-9c21-4b76-ab58-7a5854345180 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:55.373186 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ae5f4-9c21-4b76-ab58-7a5854345180 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:55.373465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ae5f4-9c21-4b76-ab58-7a5854345180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:55.374208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ae5f4-9c21-4b76-ab58-7a5854345180 None admin] RBAC: Adding query filter params (role.id=e0822035b245457eade1325bbe888913, scope.project.id=1ee18025f6d34d0daf407c44c5f28280, user.id=929b1a056a8f439387ff4b804edcbb0e) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:55.374521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ae5f4-9c21-4b76-ab58-7a5854345180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:55.375993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ae5f4-9c21-4b76-ab58-7a5854345180 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:55.382152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 97/202] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:55 2022] GET /identity/v3/role_assignments?user.id=929b1a056a8f439387ff4b804edcbb0e&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=e0822035b245457eade1325bbe888913 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:55.576968 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32e297-7ad9-4950-8507-f96391e07352 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:55.685631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32e297-7ad9-4950-8507-f96391e07352 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:55.704506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:55.723749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a70358-8b57-440d-bf1b-aced4f7bd6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:55.724459 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70358-8b57-440d-bf1b-aced4f7bd6ce None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:55.724802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70358-8b57-440d-bf1b-aced4f7bd6ce None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:55.725083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70358-8b57-440d-bf1b-aced4f7bd6ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:55.736662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a70358-8b57-440d-bf1b-aced4f7bd6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-b3a70358-8b57-440d-bf1b-aced4f7bd6ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:55.741744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:55.742654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:55.743456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 106/203] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:55.749110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ae5f4-9c21-4b76-ab58-7a5854345180 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:55.865985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:55.890480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ae5f4-9c21-4b76-ab58-7a5854345180 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:55.933528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c796131-1d12-4830-96e1-4150f347eccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:55.934150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c796131-1d12-4830-96e1-4150f347eccf None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:55.934433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c796131-1d12-4830-96e1-4150f347eccf None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:55.934710 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c796131-1d12-4830-96e1-4150f347eccf None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:55.935389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c796131-1d12-4830-96e1-4150f347eccf None admin] RBAC: Adding query filter params (name=reader) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:55.935696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c796131-1d12-4830-96e1-4150f347eccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:55.937101 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c796131-1d12-4830-96e1-4150f347eccf None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:55.947907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 98/204] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:30:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:55.953825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a70358-8b57-440d-bf1b-aced4f7bd6ce None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:56.106324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a70358-8b57-440d-bf1b-aced4f7bd6ce None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:56.152748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48301bb-f429-47a8-bde5-bf3474ef9416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:56.153286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48301bb-f429-47a8-bde5-bf3474ef9416 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:56.153566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48301bb-f429-47a8-bde5-bf3474ef9416 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:56.153836 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48301bb-f429-47a8-bde5-bf3474ef9416 None admin] PATH_INFO: `/v3/users/6fde9f2e823947639d9cd093fe859f57` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:56.154976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48301bb-f429-47a8-bde5-bf3474ef9416 None admin] RBAC: Authorizing `identity:get_user(user_id=6fde9f2e823947639d9cd093fe859f57)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:56.156481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48301bb-f429-47a8-bde5-bf3474ef9416 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:56.158045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 107/205] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:55 2022] GET /identity/v3/users/6fde9f2e823947639d9cd093fe859f57 => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:56.164512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c796131-1d12-4830-96e1-4150f347eccf None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:56.381612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c796131-1d12-4830-96e1-4150f347eccf None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:56.428134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6533f6d-1f90-4c0a-90ff-5ac65a20743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:56.428134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6533f6d-1f90-4c0a-90ff-5ac65a20743a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:56.428134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6533f6d-1f90-4c0a-90ff-5ac65a20743a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:56.428134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6533f6d-1f90-4c0a-90ff-5ac65a20743a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:56.446272 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:56.519900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6533f6d-1f90-4c0a-90ff-5ac65a20743a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:56.521376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6533f6d-1f90-4c0a-90ff-5ac65a20743a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:56.654100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 99/206] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 15:30:56.660947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48301bb-f429-47a8-bde5-bf3474ef9416 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:56.751681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:56.821793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48301bb-f429-47a8-bde5-bf3474ef9416 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:56.860407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48e4c4-fd8d-490a-9e61-46fdd5497773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:56.861028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48e4c4-fd8d-490a-9e61-46fdd5497773 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:56.861306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48e4c4-fd8d-490a-9e61-46fdd5497773 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:56.861581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48e4c4-fd8d-490a-9e61-46fdd5497773 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:56.862728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48e4c4-fd8d-490a-9e61-46fdd5497773 None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:56.864202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48e4c4-fd8d-490a-9e61-46fdd5497773 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:56.866381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 108/207] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:56 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:56.872497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6533f6d-1f90-4c0a-90ff-5ac65a20743a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:56.937268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:56.990975 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6533f6d-1f90-4c0a-90ff-5ac65a20743a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:57.025427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23ed23-d1f7-42b4-b7f1-25628f5c1e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:57.025953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23ed23-d1f7-42b4-b7f1-25628f5c1e44 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:57.026233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23ed23-d1f7-42b4-b7f1-25628f5c1e44 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:57.026505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23ed23-d1f7-42b4-b7f1-25628f5c1e44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:57.027235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23ed23-d1f7-42b4-b7f1-25628f5c1e44 None admin] RBAC: Adding query filter params (role.id=1c9432f222d8471bb13d0479297cf5d5, scope.project.id=f1c8960e806347658b410f7278fe7ffd, user.id=6fde9f2e823947639d9cd093fe859f57) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:57.027541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23ed23-d1f7-42b4-b7f1-25628f5c1e44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:57.029006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23ed23-d1f7-42b4-b7f1-25628f5c1e44 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:57.038874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 100/208] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:56 2022] GET /identity/v3/role_assignments?user.id=6fde9f2e823947639d9cd093fe859f57&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=1c9432f222d8471bb13d0479297cf5d5 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:30:57.433350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114ba48-d01f-48a4-80b6-4fb2de4192c0 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:57.433350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114ba48-d01f-48a4-80b6-4fb2de4192c0 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:57.433350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114ba48-d01f-48a4-80b6-4fb2de4192c0 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:57.433350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 109/209] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:57.433704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d26a5-ea4c-490f-b6e1-e73235e32050 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:57.433704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d26a5-ea4c-490f-b6e1-e73235e32050 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:57.433704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d26a5-ea4c-490f-b6e1-e73235e32050 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:57.433899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f69e85-4e22-4afa-937e-8ef7f923e2c8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:57.434225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f69e85-4e22-4afa-937e-8ef7f923e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:57.434320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 101/210] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:57.434495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f69e85-4e22-4afa-937e-8ef7f923e2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:57.439586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8deb1-5fc6-42f2-8352-6be4d59b2bfe None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:57.439881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8deb1-5fc6-42f2-8352-6be4d59b2bfe None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:57.440163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8deb1-5fc6-42f2-8352-6be4d59b2bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:57.454673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-25f69e85-4e22-4afa-937e-8ef7f923e2c8 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:57.455909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-25f69e85-4e22-4afa-937e-8ef7f923e2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:57.459236 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-d9e8deb1-5fc6-42f2-8352-6be4d59b2bfe None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:57.460504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d9e8deb1-5fc6-42f2-8352-6be4d59b2bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:57.533221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 110/211] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:30:57.534498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2001cb-dda3-4b6f-a8ba-85d8ca5fbca7 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:57.534788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2001cb-dda3-4b6f-a8ba-85d8ca5fbca7 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:57.535068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2001cb-dda3-4b6f-a8ba-85d8ca5fbca7 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:57.538528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 111/212] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:57.544271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea0435-f586-48f9-8aad-8a4008294441 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:57.544560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea0435-f586-48f9-8aad-8a4008294441 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:57.544837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea0435-f586-48f9-8aad-8a4008294441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:57.546977 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 102/213] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:30:57.564240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f2ea0435-f586-48f9-8aad-8a4008294441 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:57.565463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f2ea0435-f586-48f9-8aad-8a4008294441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:57.613591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:57.642521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 112/214] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:57.724893 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6c58a-e2e9-468e-a6f8-98c12b42425e None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:57.725058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6c58a-e2e9-468e-a6f8-98c12b42425e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:57.725439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6c58a-e2e9-468e-a6f8-98c12b42425e None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:57.726900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 103/215] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:30:57.732106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee32c4-6e5b-4913-bc4f-8ead56957c45 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:57.732478 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee32c4-6e5b-4913-bc4f-8ead56957c45 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:57.732834 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee32c4-6e5b-4913-bc4f-8ead56957c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:57.751260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-13ee32c4-6e5b-4913-bc4f-8ead56957c45 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:30:57.752485 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-13ee32c4-6e5b-4913-bc4f-8ead56957c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:30:57.821020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 113/216] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:30:57.850667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6c58a-e2e9-468e-a6f8-98c12b42425e None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:57.852904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee32c4-6e5b-4913-bc4f-8ead56957c45 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:57.895327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:57.958640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee32c4-6e5b-4913-bc4f-8ead56957c45 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:57.961165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:57.963215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6c58a-e2e9-468e-a6f8-98c12b42425e None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:57.996765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec15cff-76c4-4e28-9086-20fb2d4f3861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:57.997316 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec15cff-76c4-4e28-9086-20fb2d4f3861 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:57.997603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec15cff-76c4-4e28-9086-20fb2d4f3861 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:57.997882 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec15cff-76c4-4e28-9086-20fb2d4f3861 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:58.005507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241c2eb-c627-423d-8dcb-b27c9f030170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:58.006162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241c2eb-c627-423d-8dcb-b27c9f030170 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:58.006449 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241c2eb-c627-423d-8dcb-b27c9f030170 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:58.006731 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241c2eb-c627-423d-8dcb-b27c9f030170 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:58.009533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec15cff-76c4-4e28-9086-20fb2d4f3861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-bec15cff-76c4-4e28-9086-20fb2d4f3861 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:58.014675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:58.015766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:30:58.016811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:58.020274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241c2eb-c627-423d-8dcb-b27c9f030170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:58.022785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 114/217] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:30:58.023625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec15cff-76c4-4e28-9086-20fb2d4f3861 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-a241c2eb-c627-423d-8dcb-b27c9f030170 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:58.024964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:58.025823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:30:58.026631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 104/218] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:58.027322 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241c2eb-c627-423d-8dcb-b27c9f030170 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:58.130062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:58.133103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241c2eb-c627-423d-8dcb-b27c9f030170 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:58.142159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec15cff-76c4-4e28-9086-20fb2d4f3861 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:58.167371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b00024-b3ff-4e75-a944-dd2a004bdb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:58.168028 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b00024-b3ff-4e75-a944-dd2a004bdb7b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:58.168314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b00024-b3ff-4e75-a944-dd2a004bdb7b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:58.168589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b00024-b3ff-4e75-a944-dd2a004bdb7b None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:58.169262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b00024-b3ff-4e75-a944-dd2a004bdb7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:58.169567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b00024-b3ff-4e75-a944-dd2a004bdb7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:58.170951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b00024-b3ff-4e75-a944-dd2a004bdb7b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:58.176627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7c178-bd32-4817-a884-4f453cb23b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:58.177264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7c178-bd32-4817-a884-4f453cb23b74 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:58.177547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7c178-bd32-4817-a884-4f453cb23b74 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:58.177825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7c178-bd32-4817-a884-4f453cb23b74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:58.179522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7c178-bd32-4817-a884-4f453cb23b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:58.179777 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 105/219] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:30:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:58.180687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b00024-b3ff-4e75-a944-dd2a004bdb7b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:58.184475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7c178-bd32-4817-a884-4f453cb23b74 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:58.186213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 115/220] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:30:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:58.186858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7c178-bd32-4817-a884-4f453cb23b74 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:58.218216 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:58.290233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b00024-b3ff-4e75-a944-dd2a004bdb7b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:58.292308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7c178-bd32-4817-a884-4f453cb23b74 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:58.326007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a6775-52ea-4050-aaa4-c1a6153aeb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:58.326622 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a6775-52ea-4050-aaa4-c1a6153aeb34 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:58.326909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a6775-52ea-4050-aaa4-c1a6153aeb34 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:58.327188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a6775-52ea-4050-aaa4-c1a6153aeb34 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:58.327854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a6775-52ea-4050-aaa4-c1a6153aeb34 None admin] RBAC: Adding query filter params (name=member) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:58.328191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a6775-52ea-4050-aaa4-c1a6153aeb34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:58.329200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a25b0-1720-403d-8f79-163654dcd027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:58.329698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a6775-52ea-4050-aaa4-c1a6153aeb34 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:58.329817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a25b0-1720-403d-8f79-163654dcd027 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:58.330098 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a25b0-1720-403d-8f79-163654dcd027 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:58.330369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a25b0-1720-403d-8f79-163654dcd027 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:58.337988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 106/221] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:30:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:58.338824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a6775-52ea-4050-aaa4-c1a6153aeb34 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:58.340500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a25b0-1720-403d-8f79-163654dcd027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-ad5a25b0-1720-403d-8f79-163654dcd027 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:30:58.345384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:30:58.346350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:30:58.347163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 116/222] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:30:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:30:58.347969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a25b0-1720-403d-8f79-163654dcd027 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:58.375732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:58.442236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:58.450630 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a25b0-1720-403d-8f79-163654dcd027 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:58.451816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a6775-52ea-4050-aaa4-c1a6153aeb34 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:58.485874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f462d80-2c24-4867-a29c-a81915c5fc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:58.486253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd5ebb-6017-4b3d-b74c-f48698d7238b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:58.486484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f462d80-2c24-4867-a29c-a81915c5fc9d None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:58.486772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f462d80-2c24-4867-a29c-a81915c5fc9d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:58.486886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd5ebb-6017-4b3d-b74c-f48698d7238b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:58.487054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f462d80-2c24-4867-a29c-a81915c5fc9d None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:58.487168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd5ebb-6017-4b3d-b74c-f48698d7238b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:58.487444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd5ebb-6017-4b3d-b74c-f48698d7238b None admin] PATH_INFO: `/v3/users/929b1a056a8f439387ff4b804edcbb0e` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:58.487702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f462d80-2c24-4867-a29c-a81915c5fc9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:58.488694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd5ebb-6017-4b3d-b74c-f48698d7238b None admin] RBAC: Authorizing `identity:get_user(user_id=929b1a056a8f439387ff4b804edcbb0e)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:58.489177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f462d80-2c24-4867-a29c-a81915c5fc9d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:58.490115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd5ebb-6017-4b3d-b74c-f48698d7238b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:58.491701 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 107/223] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:58 2022] GET /identity/v3/users/929b1a056a8f439387ff4b804edcbb0e => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:58.492485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd5ebb-6017-4b3d-b74c-f48698d7238b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:58.583150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:58.587760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 117/224] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:30:58 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:30:58.588644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f462d80-2c24-4867-a29c-a81915c5fc9d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:58.597010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd5ebb-6017-4b3d-b74c-f48698d7238b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:58.639540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24f6b6-73a2-4a7d-9ada-32b8aaa8ccbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:58.640237 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24f6b6-73a2-4a7d-9ada-32b8aaa8ccbd None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:58.640526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24f6b6-73a2-4a7d-9ada-32b8aaa8ccbd None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:58.640816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24f6b6-73a2-4a7d-9ada-32b8aaa8ccbd None admin] PATH_INFO: `/v3/users/a8d5924fb50340998b79d9061e2bfc28` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:58.659578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24f6b6-73a2-4a7d-9ada-32b8aaa8ccbd None admin] RBAC: Authorizing `identity:get_user(user_id=a8d5924fb50340998b79d9061e2bfc28)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:58.661103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24f6b6-73a2-4a7d-9ada-32b8aaa8ccbd None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:58.663487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 108/225] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:58 2022] GET /identity/v3/users/a8d5924fb50340998b79d9061e2bfc28 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:58.664340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24f6b6-73a2-4a7d-9ada-32b8aaa8ccbd None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:58.726416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f462d80-2c24-4867-a29c-a81915c5fc9d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:58.775366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bb9bb-5927-4288-8321-ff7c092fc18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:58.776269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bb9bb-5927-4288-8321-ff7c092fc18f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:58.776552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bb9bb-5927-4288-8321-ff7c092fc18f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:58.776831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bb9bb-5927-4288-8321-ff7c092fc18f None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:58.777954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bb9bb-5927-4288-8321-ff7c092fc18f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:58.778323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bb9bb-5927-4288-8321-ff7c092fc18f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:58.779998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bb9bb-5927-4288-8321-ff7c092fc18f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:58.780470 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24f6b6-73a2-4a7d-9ada-32b8aaa8ccbd None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:58.793109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 118/226] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:30:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:58.793958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bb9bb-5927-4288-8321-ff7c092fc18f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:58.812406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:58.821932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7061f27-5e40-4508-bfd3-94ebd6031d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:58.822577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7061f27-5e40-4508-bfd3-94ebd6031d25 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:58.822852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7061f27-5e40-4508-bfd3-94ebd6031d25 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:58.823240 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7061f27-5e40-4508-bfd3-94ebd6031d25 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:58.824304 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7061f27-5e40-4508-bfd3-94ebd6031d25 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:58.825748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7061f27-5e40-4508-bfd3-94ebd6031d25 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:58.827738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 109/227] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:58 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:58.828597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7061f27-5e40-4508-bfd3-94ebd6031d25 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:58.854836 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:58.922065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bb9bb-5927-4288-8321-ff7c092fc18f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:58.956397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7061f27-5e40-4508-bfd3-94ebd6031d25 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:58.962565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe53a0f-6744-4380-bdba-5767dc9bc236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:58.963568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe53a0f-6744-4380-bdba-5767dc9bc236 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:58.964102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe53a0f-6744-4380-bdba-5767dc9bc236 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:58.965141 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe53a0f-6744-4380-bdba-5767dc9bc236 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:59.002217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd8c6c-3c1d-474d-b3b0-4ffcf2cf2178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:59.002962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd8c6c-3c1d-474d-b3b0-4ffcf2cf2178 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:59.003256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd8c6c-3c1d-474d-b3b0-4ffcf2cf2178 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:59.003530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd8c6c-3c1d-474d-b3b0-4ffcf2cf2178 None admin] PATH_INFO: `/v3/users/929b1a056a8f439387ff4b804edcbb0e` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:59.005004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd8c6c-3c1d-474d-b3b0-4ffcf2cf2178 None admin] RBAC: Authorizing `identity:get_user(user_id=929b1a056a8f439387ff4b804edcbb0e)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:59.007460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd8c6c-3c1d-474d-b3b0-4ffcf2cf2178 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:59.013386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 110/228] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:30:58 2022] GET /identity/v3/users/929b1a056a8f439387ff4b804edcbb0e => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:59.013386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd8c6c-3c1d-474d-b3b0-4ffcf2cf2178 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:59.064940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe53a0f-6744-4380-bdba-5767dc9bc236 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:59.066802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe53a0f-6744-4380-bdba-5767dc9bc236 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:59.080370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:59.118388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:59.135279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd8c6c-3c1d-474d-b3b0-4ffcf2cf2178 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:59.170934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353f941-d447-4d4d-9827-cc95b73d2a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:59.171997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353f941-d447-4d4d-9827-cc95b73d2a27 None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:59.172582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353f941-d447-4d4d-9827-cc95b73d2a27 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:59.173140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353f941-d447-4d4d-9827-cc95b73d2a27 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280/users/929b1a056a8f439387ff4b804edcbb0e/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:59.175317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353f941-d447-4d4d-9827-cc95b73d2a27 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee18025f6d34d0daf407c44c5f28280, user_id=929b1a056a8f439387ff4b804edcbb0e, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:59.178765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353f941-d447-4d4d-9827-cc95b73d2a27 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:59.184658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 119/229] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:30:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:30:59.184658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe53a0f-6744-4380-bdba-5767dc9bc236 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:59.289661 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 111/230] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:30:59 2022] PUT /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280/users/929b1a056a8f439387ff4b804edcbb0e/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:30:59.290750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353f941-d447-4d4d-9827-cc95b73d2a27 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:59.298645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe53a0f-6744-4380-bdba-5767dc9bc236 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:59.322196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:59.372776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237162db-09d1-4cb6-85ce-6f4302cf041f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:59.373342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237162db-09d1-4cb6-85ce-6f4302cf041f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:59.373628 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237162db-09d1-4cb6-85ce-6f4302cf041f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:59.373908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237162db-09d1-4cb6-85ce-6f4302cf041f None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:59.376876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237162db-09d1-4cb6-85ce-6f4302cf041f None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:59.378527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237162db-09d1-4cb6-85ce-6f4302cf041f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:59.381880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 120/231] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:59 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:59.387436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237162db-09d1-4cb6-85ce-6f4302cf041f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:59.523824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353f941-d447-4d4d-9827-cc95b73d2a27 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:59.530716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237162db-09d1-4cb6-85ce-6f4302cf041f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:59.573183 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df66ca-163e-41fa-8152-d6877f835260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:59.574164 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df66ca-163e-41fa-8152-d6877f835260 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:59.574459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df66ca-163e-41fa-8152-d6877f835260 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:59.576246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df66ca-163e-41fa-8152-d6877f835260 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:59.577737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df66ca-163e-41fa-8152-d6877f835260 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:59.580071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df66ca-163e-41fa-8152-d6877f835260 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:59.584667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f67f74-cfdc-4416-b702-3b39a0eb7690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:59.585346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f67f74-cfdc-4416-b702-3b39a0eb7690 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:59.585634 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f67f74-cfdc-4416-b702-3b39a0eb7690 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:59.585912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f67f74-cfdc-4416-b702-3b39a0eb7690 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280/users/929b1a056a8f439387ff4b804edcbb0e/roles/e0822035b245457eade1325bbe888913` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:59.586333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 112/232] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:30:59 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:30:59.591924 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f67f74-cfdc-4416-b702-3b39a0eb7690 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee18025f6d34d0daf407c44c5f28280, user_id=929b1a056a8f439387ff4b804edcbb0e, role_id=e0822035b245457eade1325bbe888913)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:59.591924 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f67f74-cfdc-4416-b702-3b39a0eb7690 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:59.593195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df66ca-163e-41fa-8152-d6877f835260 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:30:59.620527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:30:59.696426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 121/233] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:30:59 2022] PUT /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280/users/929b1a056a8f439387ff4b804edcbb0e/roles/e0822035b245457eade1325bbe888913 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:30:59.717905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df66ca-163e-41fa-8152-d6877f835260 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:30:59.759006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f07035-5272-4110-9e50-7112cc5d90ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:30:59.759651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07035-5272-4110-9e50-7112cc5d90ba None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:30:59.759976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07035-5272-4110-9e50-7112cc5d90ba None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:30:59.760257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07035-5272-4110-9e50-7112cc5d90ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:30:59.761081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f07035-5272-4110-9e50-7112cc5d90ba None admin] RBAC: Adding query filter params (role.id=0166494127b14615900b2c121908b2ba, scope.project.id=1ee18025f6d34d0daf407c44c5f28280, user.id=a8d5924fb50340998b79d9061e2bfc28) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:30:59.761403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f07035-5272-4110-9e50-7112cc5d90ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:30:59.762925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f07035-5272-4110-9e50-7112cc5d90ba None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:30:59.769602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 113/234] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:30:59 2022] GET /identity/v3/role_assignments?user.id=a8d5924fb50340998b79d9061e2bfc28&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=0166494127b14615900b2c121908b2ba => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:00.829991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91551c4-6b4d-4554-9fe4-7dcdf3db567d None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:00.830520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91551c4-6b4d-4554-9fe4-7dcdf3db567d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:00.830520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91551c4-6b4d-4554-9fe4-7dcdf3db567d None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:00.831908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 122/235] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:00.837279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dccce3-350f-4ba1-9174-a7ae9584c1c2 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:00.837661 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dccce3-350f-4ba1-9174-a7ae9584c1c2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:00.837845 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dccce3-350f-4ba1-9174-a7ae9584c1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:00.864831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-45dccce3-350f-4ba1-9174-a7ae9584c1c2 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:00.866519 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-45dccce3-350f-4ba1-9174-a7ae9584c1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:00.922737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac40d8-8d2e-48bc-8911-b5ec559e1d61 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:00.923058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac40d8-8d2e-48bc-8911-b5ec559e1d61 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:00.923320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac40d8-8d2e-48bc-8911-b5ec559e1d61 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:00.924661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 123/236] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:00.930434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29702379-5a48-41dc-953f-6a3e41056ef6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:00.930696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29702379-5a48-41dc-953f-6a3e41056ef6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:00.930972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29702379-5a48-41dc-953f-6a3e41056ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:00.948714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-29702379-5a48-41dc-953f-6a3e41056ef6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:00.949894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-29702379-5a48-41dc-953f-6a3e41056ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:00.974876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 114/237] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:01.008471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:01.060385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 124/238] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:01.360105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dccce3-350f-4ba1-9174-a7ae9584c1c2 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:01.374302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:01.395160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29702379-5a48-41dc-953f-6a3e41056ef6 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:01.527299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29702379-5a48-41dc-953f-6a3e41056ef6 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:01.565566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d72d0a-5a76-4bcd-8ea7-3d1574151d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:01.566208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d72d0a-5a76-4bcd-8ea7-3d1574151d28 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:01.566495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d72d0a-5a76-4bcd-8ea7-3d1574151d28 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:01.566775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d72d0a-5a76-4bcd-8ea7-3d1574151d28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:01.584201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d72d0a-5a76-4bcd-8ea7-3d1574151d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-06d72d0a-5a76-4bcd-8ea7-3d1574151d28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:01.589958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:01.590916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:01.591734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:01.593089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 125/239] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:01.601649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d72d0a-5a76-4bcd-8ea7-3d1574151d28 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:01.610549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dccce3-350f-4ba1-9174-a7ae9584c1c2 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:01.660261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:01.662207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f6fe9-aa52-4ee4-99bd-ef270e789f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:01.663117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f6fe9-aa52-4ee4-99bd-ef270e789f20 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:01.663791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f6fe9-aa52-4ee4-99bd-ef270e789f20 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:01.665494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f6fe9-aa52-4ee4-99bd-ef270e789f20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:01.667481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f6fe9-aa52-4ee4-99bd-ef270e789f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:01.669713 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f6fe9-aa52-4ee4-99bd-ef270e789f20 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:01.674611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 115/240] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:01.680911 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f6fe9-aa52-4ee4-99bd-ef270e789f20 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:01.784239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:01.795020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d72d0a-5a76-4bcd-8ea7-3d1574151d28 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:01.813605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f6fe9-aa52-4ee4-99bd-ef270e789f20 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:01.865311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697c358-eec6-4407-b9cd-a51f2da697bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:01.865877 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697c358-eec6-4407-b9cd-a51f2da697bd None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:01.866166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697c358-eec6-4407-b9cd-a51f2da697bd None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:01.866441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697c358-eec6-4407-b9cd-a51f2da697bd None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:01.867123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697c358-eec6-4407-b9cd-a51f2da697bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:01.867428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697c358-eec6-4407-b9cd-a51f2da697bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:01.869461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697c358-eec6-4407-b9cd-a51f2da697bd None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:01.874784 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bf4f0-0fa3-4e9a-8ac5-0b5bf5c26387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:01.875402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf4f0-0fa3-4e9a-8ac5-0b5bf5c26387 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:01.875687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf4f0-0fa3-4e9a-8ac5-0b5bf5c26387 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:01.875983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf4f0-0fa3-4e9a-8ac5-0b5bf5c26387 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:01.876642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bf4f0-0fa3-4e9a-8ac5-0b5bf5c26387 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:01.878057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bf4f0-0fa3-4e9a-8ac5-0b5bf5c26387 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:01.891498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 126/241] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:01.894671 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9423-5cda-472e-a33f-6c6c6f9f0f13 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:01.894958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9423-5cda-472e-a33f-6c6c6f9f0f13 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:01.895238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9423-5cda-472e-a33f-6c6c6f9f0f13 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:01.896850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 127/242] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:01.897502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9423-5cda-472e-a33f-6c6c6f9f0f13 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:01.993329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:02.004034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9423-5cda-472e-a33f-6c6c6f9f0f13 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:02.020395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 116/243] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:31:01 2022] POST /identity/v3/users => generated 293 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:31:02.022847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79daf7-0492-4f1a-afc2-adbe3873fbcd None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:02.023131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79daf7-0492-4f1a-afc2-adbe3873fbcd None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:02.023634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79daf7-0492-4f1a-afc2-adbe3873fbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:02.063112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7c79daf7-0492-4f1a-afc2-adbe3873fbcd None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:02.065247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7c79daf7-0492-4f1a-afc2-adbe3873fbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:02.073504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bbab3-556e-4cf8-bd7d-d387820b50ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:02.075951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bbab3-556e-4cf8-bd7d-d387820b50ea None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:02.076242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bbab3-556e-4cf8-bd7d-d387820b50ea None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:02.076520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bbab3-556e-4cf8-bd7d-d387820b50ea None admin] PATH_INFO: `/v3/users/b486b19644f5496fa6370ea210bcef12` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:02.106146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bbab3-556e-4cf8-bd7d-d387820b50ea None admin] RBAC: Authorizing `identity:get_user(user_id=b486b19644f5496fa6370ea210bcef12)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:02.107712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bbab3-556e-4cf8-bd7d-d387820b50ea None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:02.111828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 128/244] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:01 2022] GET /identity/v3/users/b486b19644f5496fa6370ea210bcef12 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:02.117315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bbab3-556e-4cf8-bd7d-d387820b50ea None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:02.202353 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 117/245] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:02.268196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bbab3-556e-4cf8-bd7d-d387820b50ea None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:02.304974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7066f7-02a3-4521-a36e-187379128225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:02.305986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7066f7-02a3-4521-a36e-187379128225 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:02.306469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7066f7-02a3-4521-a36e-187379128225 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:02.306943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7066f7-02a3-4521-a36e-187379128225 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:02.333570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:02.333753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e754be7-07b6-4064-bcf1-ea7b5d78872e None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:02.334295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e754be7-07b6-4064-bcf1-ea7b5d78872e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:02.334295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e754be7-07b6-4064-bcf1-ea7b5d78872e None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:02.337632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 118/246] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:02.343807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc8012-e81e-4fca-92dc-20c01cffd288 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:02.344132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc8012-e81e-4fca-92dc-20c01cffd288 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:02.344414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc8012-e81e-4fca-92dc-20c01cffd288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:02.368097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-2adc8012-e81e-4fca-92dc-20c01cffd288 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:02.369284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-2adc8012-e81e-4fca-92dc-20c01cffd288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:02.391944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7066f7-02a3-4521-a36e-187379128225 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:02.393737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7066f7-02a3-4521-a36e-187379128225 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:02.443924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:02.463931 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 119/247] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:02.498119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787783-a714-4df7-811c-40a80fe1ab4b None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:02.498457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787783-a714-4df7-811c-40a80fe1ab4b None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:02.498715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787783-a714-4df7-811c-40a80fe1ab4b None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:02.500232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 120/248] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:02.500801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32787783-a714-4df7-811c-40a80fe1ab4b None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:02.530303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 129/249] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 15:31:02.531916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57403a-0a33-4295-87eb-69c1b011f374 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:02.532434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57403a-0a33-4295-87eb-69c1b011f374 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:02.532918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57403a-0a33-4295-87eb-69c1b011f374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:02.556460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0f57403a-0a33-4295-87eb-69c1b011f374 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:02.558049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0f57403a-0a33-4295-87eb-69c1b011f374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:02.617286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:02.647048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32787783-a714-4df7-811c-40a80fe1ab4b None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:02.651694 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 130/250] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:02.652872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57403a-0a33-4295-87eb-69c1b011f374 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:02.692120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407637d-bb5b-412d-9164-a27bb36213cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:02.692879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407637d-bb5b-412d-9164-a27bb36213cc None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:02.693219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407637d-bb5b-412d-9164-a27bb36213cc None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:02.693496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407637d-bb5b-412d-9164-a27bb36213cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:02.714658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407637d-bb5b-412d-9164-a27bb36213cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-8407637d-bb5b-412d-9164-a27bb36213cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:02.719518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:02.720515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:02.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 121/251] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:31:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:02.729449 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407637d-bb5b-412d-9164-a27bb36213cc None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:02.792595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57403a-0a33-4295-87eb-69c1b011f374 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:02.805959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:02.839341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f86d4-6667-4967-b782-54b07b068808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:02.840016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f86d4-6667-4967-b782-54b07b068808 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:02.841609 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f86d4-6667-4967-b782-54b07b068808 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:02.842656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f86d4-6667-4967-b782-54b07b068808 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:02.849494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f86d4-6667-4967-b782-54b07b068808 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:02.853012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f86d4-6667-4967-b782-54b07b068808 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:02.858289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 131/252] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:02 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:02.861103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f86d4-6667-4967-b782-54b07b068808 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:02.878811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407637d-bb5b-412d-9164-a27bb36213cc None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:02.921787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:02.944642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4008c4-3f92-4b4d-a262-2d9bd60ff1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:02.945279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4008c4-3f92-4b4d-a262-2d9bd60ff1ae None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:02.945568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4008c4-3f92-4b4d-a262-2d9bd60ff1ae None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:02.945852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4008c4-3f92-4b4d-a262-2d9bd60ff1ae None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:02.946544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4008c4-3f92-4b4d-a262-2d9bd60ff1ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:02.946852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4008c4-3f92-4b4d-a262-2d9bd60ff1ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:02.948285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4008c4-3f92-4b4d-a262-2d9bd60ff1ae None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:02.969899 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 122/253] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:02.970665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4008c4-3f92-4b4d-a262-2d9bd60ff1ae None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:02.986477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f86d4-6667-4967-b782-54b07b068808 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:03.032192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c2ee5-064a-4381-89bf-a85cf35ef880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:03.033079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c2ee5-064a-4381-89bf-a85cf35ef880 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:03.033366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c2ee5-064a-4381-89bf-a85cf35ef880 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:03.034058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c2ee5-064a-4381-89bf-a85cf35ef880 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:03.048529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c2ee5-064a-4381-89bf-a85cf35ef880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-ab2c2ee5-064a-4381-89bf-a85cf35ef880 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:03.055777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:03.056703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 132/254] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:03.058344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c2ee5-064a-4381-89bf-a85cf35ef880 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:03.088593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4008c4-3f92-4b4d-a262-2d9bd60ff1ae None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:03.107573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:03.136040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02850b2-6785-4626-a1fd-ab19a521eb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:03.136886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02850b2-6785-4626-a1fd-ab19a521eb37 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:03.137552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02850b2-6785-4626-a1fd-ab19a521eb37 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:03.137972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02850b2-6785-4626-a1fd-ab19a521eb37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:03.139469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02850b2-6785-4626-a1fd-ab19a521eb37 None admin] RBAC: Adding query filter params (role.id=1c9432f222d8471bb13d0479297cf5d5, scope.project.id=1ee18025f6d34d0daf407c44c5f28280, user.id=b486b19644f5496fa6370ea210bcef12) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:03.140202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02850b2-6785-4626-a1fd-ab19a521eb37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:03.141753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02850b2-6785-4626-a1fd-ab19a521eb37 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:03.154604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 123/255] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:02 2022] GET /identity/v3/role_assignments?user.id=b486b19644f5496fa6370ea210bcef12&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=1c9432f222d8471bb13d0479297cf5d5 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:03.155693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02850b2-6785-4626-a1fd-ab19a521eb37 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:03.234039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c2ee5-064a-4381-89bf-a85cf35ef880 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:03.250954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:03.292011 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02850b2-6785-4626-a1fd-ab19a521eb37 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:03.319015 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e2d5b-7a25-4efb-9e9b-baf8ed13f96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:03.321425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e2d5b-7a25-4efb-9e9b-baf8ed13f96e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:03.322549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e2d5b-7a25-4efb-9e9b-baf8ed13f96e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:03.324117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e2d5b-7a25-4efb-9e9b-baf8ed13f96e None admin] PATH_INFO: `/v3/users/929b1a056a8f439387ff4b804edcbb0e` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:03.326120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e2d5b-7a25-4efb-9e9b-baf8ed13f96e None admin] RBAC: Authorizing `identity:get_user(user_id=929b1a056a8f439387ff4b804edcbb0e)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:03.328759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e2d5b-7a25-4efb-9e9b-baf8ed13f96e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:03.333404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 133/256] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:03 2022] GET /identity/v3/users/929b1a056a8f439387ff4b804edcbb0e => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:03.335339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e2d5b-7a25-4efb-9e9b-baf8ed13f96e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:03.371586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb83c0-81b8-4a5c-8e45-54bb930f81c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:03.373357 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb83c0-81b8-4a5c-8e45-54bb930f81c5 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:03.373463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb83c0-81b8-4a5c-8e45-54bb930f81c5 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:03.373723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb83c0-81b8-4a5c-8e45-54bb930f81c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:03.395324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb83c0-81b8-4a5c-8e45-54bb930f81c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-cdfb83c0-81b8-4a5c-8e45-54bb930f81c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:03.400600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:03.401481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:03.402346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 124/257] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:31:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:03.403058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb83c0-81b8-4a5c-8e45-54bb930f81c5 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:03.449789 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:03.513507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb83c0-81b8-4a5c-8e45-54bb930f81c5 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:03.548887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f10b1-36b8-4cf0-847f-306b8a5ca4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:03.549506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f10b1-36b8-4cf0-847f-306b8a5ca4fe None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:03.549867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f10b1-36b8-4cf0-847f-306b8a5ca4fe None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:03.550064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f10b1-36b8-4cf0-847f-306b8a5ca4fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:03.609845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e2d5b-7a25-4efb-9e9b-baf8ed13f96e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:03.622729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f10b1-36b8-4cf0-847f-306b8a5ca4fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:03.624233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f10b1-36b8-4cf0-847f-306b8a5ca4fe None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:03.665455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:03.708595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c268f63-7be3-4d67-9144-87706385e415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:03.709232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c268f63-7be3-4d67-9144-87706385e415 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:03.709519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c268f63-7be3-4d67-9144-87706385e415 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:03.710336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c268f63-7be3-4d67-9144-87706385e415 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:03.711040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c268f63-7be3-4d67-9144-87706385e415 None admin] RBAC: Adding query filter params (name=member) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:03.711341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c268f63-7be3-4d67-9144-87706385e415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:03.712809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c268f63-7be3-4d67-9144-87706385e415 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:03.730725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 125/258] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:03.731311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 134/259] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:03.732328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f10b1-36b8-4cf0-847f-306b8a5ca4fe None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:03.736579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c268f63-7be3-4d67-9144-87706385e415 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:03.832339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:03.900570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f10b1-36b8-4cf0-847f-306b8a5ca4fe None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:03.943794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f54ef-b496-4a9f-b9f4-f275023e4d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:03.944760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f54ef-b496-4a9f-b9f4-f275023e4d58 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:03.945250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f54ef-b496-4a9f-b9f4-f275023e4d58 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:03.945703 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f54ef-b496-4a9f-b9f4-f275023e4d58 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:03.946558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f54ef-b496-4a9f-b9f4-f275023e4d58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:03.947040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f54ef-b496-4a9f-b9f4-f275023e4d58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:03.948695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f54ef-b496-4a9f-b9f4-f275023e4d58 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:03.955657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:03.959552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 126/260] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:03.962007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f54ef-b496-4a9f-b9f4-f275023e4d58 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:03.978008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c268f63-7be3-4d67-9144-87706385e415 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:04.044619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f4c60-49f9-41b2-9890-8aa94069db73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:04.045264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f4c60-49f9-41b2-9890-8aa94069db73 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:04.045547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f4c60-49f9-41b2-9890-8aa94069db73 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:04.045822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f4c60-49f9-41b2-9890-8aa94069db73 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:04.047206 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f4c60-49f9-41b2-9890-8aa94069db73 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:04.048707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f4c60-49f9-41b2-9890-8aa94069db73 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:04.050618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 135/261] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:03 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:04.051645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f4c60-49f9-41b2-9890-8aa94069db73 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:04.142237 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f54ef-b496-4a9f-b9f4-f275023e4d58 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:04.209662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bacef6-ccd9-4a81-8131-6b3832e9cd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:04.210307 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bacef6-ccd9-4a81-8131-6b3832e9cd09 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:04.210587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bacef6-ccd9-4a81-8131-6b3832e9cd09 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:04.210860 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bacef6-ccd9-4a81-8131-6b3832e9cd09 None admin] PATH_INFO: `/v3/users/a8d5924fb50340998b79d9061e2bfc28` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:04.213085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bacef6-ccd9-4a81-8131-6b3832e9cd09 None admin] RBAC: Authorizing `identity:get_user(user_id=a8d5924fb50340998b79d9061e2bfc28)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:04.214527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bacef6-ccd9-4a81-8131-6b3832e9cd09 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:04.216119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 127/262] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:03 2022] GET /identity/v3/users/a8d5924fb50340998b79d9061e2bfc28 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:04.217223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bacef6-ccd9-4a81-8131-6b3832e9cd09 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:04.220797 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f4c60-49f9-41b2-9890-8aa94069db73 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:04.228205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:04.282047 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb6f7b-2709-4465-8e93-9e3f03b21e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:04.282689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb6f7b-2709-4465-8e93-9e3f03b21e3d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:04.282978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb6f7b-2709-4465-8e93-9e3f03b21e3d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:04.283259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb6f7b-2709-4465-8e93-9e3f03b21e3d None admin] PATH_INFO: `/v3/users/929b1a056a8f439387ff4b804edcbb0e` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:04.286463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb6f7b-2709-4465-8e93-9e3f03b21e3d None admin] RBAC: Authorizing `identity:get_user(user_id=929b1a056a8f439387ff4b804edcbb0e)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:04.288488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb6f7b-2709-4465-8e93-9e3f03b21e3d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:04.290102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 136/263] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:04 2022] GET /identity/v3/users/929b1a056a8f439387ff4b804edcbb0e => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:04.292703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb6f7b-2709-4465-8e93-9e3f03b21e3d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:04.325045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:04.360567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bacef6-ccd9-4a81-8131-6b3832e9cd09 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:04.399731 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c368e-ac30-4aaa-9e09-1b410260802f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:04.400478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c368e-ac30-4aaa-9e09-1b410260802f None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:04.400772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c368e-ac30-4aaa-9e09-1b410260802f None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:04.401052 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c368e-ac30-4aaa-9e09-1b410260802f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:04.401824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c368e-ac30-4aaa-9e09-1b410260802f None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=1ee18025f6d34d0daf407c44c5f28280, user.id=929b1a056a8f439387ff4b804edcbb0e) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:04.402142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c368e-ac30-4aaa-9e09-1b410260802f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:04.403655 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c368e-ac30-4aaa-9e09-1b410260802f None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:04.410743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 128/264] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:04 2022] GET /identity/v3/role_assignments?user.id=929b1a056a8f439387ff4b804edcbb0e&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=6ba5a5add2174983be37258aad1ee404 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:04.411668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c368e-ac30-4aaa-9e09-1b410260802f None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:04.580485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c368e-ac30-4aaa-9e09-1b410260802f None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:04.640361 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:04.648570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be92bee-7627-44c7-850c-542aab8a70e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:04.649192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be92bee-7627-44c7-850c-542aab8a70e0 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:04.649482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be92bee-7627-44c7-850c-542aab8a70e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:04.649915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be92bee-7627-44c7-850c-542aab8a70e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:04.680350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb6f7b-2709-4465-8e93-9e3f03b21e3d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:04.754940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be92bee-7627-44c7-850c-542aab8a70e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:04.756646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be92bee-7627-44c7-850c-542aab8a70e0 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:04.767302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfbb62-6f5c-4bcc-ad9e-6299e8d8b939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:04.768318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfbb62-6f5c-4bcc-ad9e-6299e8d8b939 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:04.768804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfbb62-6f5c-4bcc-ad9e-6299e8d8b939 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:04.769304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfbb62-6f5c-4bcc-ad9e-6299e8d8b939 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:04.770756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfbb62-6f5c-4bcc-ad9e-6299e8d8b939 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:04.772570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfbb62-6f5c-4bcc-ad9e-6299e8d8b939 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:04.774487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 137/265] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:04 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 484 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:04.789844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfbb62-6f5c-4bcc-ad9e-6299e8d8b939 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:04.855755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:04.889744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 129/266] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:04.890910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f5548-e44e-48c7-a8e6-629ead6b0524 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:04.891276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f5548-e44e-48c7-a8e6-629ead6b0524 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:04.891563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f5548-e44e-48c7-a8e6-629ead6b0524 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:04.893288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 130/267] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:04.894094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f5548-e44e-48c7-a8e6-629ead6b0524 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:04.932448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:04.937534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfbb62-6f5c-4bcc-ad9e-6299e8d8b939 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:04.982817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbf971-03b5-4ab9-a430-a043f34adc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:04.983498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbf971-03b5-4ab9-a430-a043f34adc1f None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:04.983781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbf971-03b5-4ab9-a430-a043f34adc1f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:04.984198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbf971-03b5-4ab9-a430-a043f34adc1f None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280/users/a8d5924fb50340998b79d9061e2bfc28/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:04.986206 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbf971-03b5-4ab9-a430-a043f34adc1f None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee18025f6d34d0daf407c44c5f28280, user_id=a8d5924fb50340998b79d9061e2bfc28, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:04.987919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbf971-03b5-4ab9-a430-a043f34adc1f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:05.018424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f5548-e44e-48c7-a8e6-629ead6b0524 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:05.053219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309f2c5-e643-4fe5-b350-47d5323d40a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:05.053858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309f2c5-e643-4fe5-b350-47d5323d40a7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:05.054143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309f2c5-e643-4fe5-b350-47d5323d40a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:05.054422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309f2c5-e643-4fe5-b350-47d5323d40a7 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:05.055553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309f2c5-e643-4fe5-b350-47d5323d40a7 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:05.057069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309f2c5-e643-4fe5-b350-47d5323d40a7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:05.058924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 131/268] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:04 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:05.063506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 138/269] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:31:04 2022] PUT /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280/users/a8d5924fb50340998b79d9061e2bfc28/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:31:05.064946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7068-a099-4799-a276-735999d00df6 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:05.065243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7068-a099-4799-a276-735999d00df6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:05.065525 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7068-a099-4799-a276-735999d00df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:05.070170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd4be2-f531-4405-be0b-13864494a0fd None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:05.070473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd4be2-f531-4405-be0b-13864494a0fd None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:05.070729 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd4be2-f531-4405-be0b-13864494a0fd None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:05.080347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 139/270] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:05 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:05.081243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd4be2-f531-4405-be0b-13864494a0fd None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:05.094473 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-0d4b7068-a099-4799-a276-735999d00df6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:05.097200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0d4b7068-a099-4799-a276-735999d00df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:05.204634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 132/271] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:05.206674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92c25d-517b-454b-80d7-8abf192d480e None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:05.207184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92c25d-517b-454b-80d7-8abf192d480e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:05.208185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92c25d-517b-454b-80d7-8abf192d480e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:05.212138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:05.220894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd4be2-f531-4405-be0b-13864494a0fd None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:05.232076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ac92c25d-517b-454b-80d7-8abf192d480e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:05.233664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ac92c25d-517b-454b-80d7-8abf192d480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:05.261592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:05.271023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561369fa-0ac0-401e-a1e2-8c68566f78c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:05.271924 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561369fa-0ac0-401e-a1e2-8c68566f78c8 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:05.272933 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561369fa-0ac0-401e-a1e2-8c68566f78c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:05.273394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561369fa-0ac0-401e-a1e2-8c68566f78c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:05.274410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561369fa-0ac0-401e-a1e2-8c68566f78c8 None admin] RBAC: Adding query filter params (role.id=e0822035b245457eade1325bbe888913, scope.project.id=1ee18025f6d34d0daf407c44c5f28280, user.id=929b1a056a8f439387ff4b804edcbb0e) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:05.274908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561369fa-0ac0-401e-a1e2-8c68566f78c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:05.277620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561369fa-0ac0-401e-a1e2-8c68566f78c8 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:05.285610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 140/272] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:05 2022] GET /identity/v3/role_assignments?user.id=929b1a056a8f439387ff4b804edcbb0e&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=e0822035b245457eade1325bbe888913 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:05.320775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 133/273] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:05.455791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75d825-1ac4-4f24-b096-526de3388901 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:05.456370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75d825-1ac4-4f24-b096-526de3388901 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:05.456864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75d825-1ac4-4f24-b096-526de3388901 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:05.458393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 141/274] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:05.465729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e5fe-253e-4ee1-a746-439289d46f81 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:05.466100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e5fe-253e-4ee1-a746-439289d46f81 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:05.466459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e5fe-253e-4ee1-a746-439289d46f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:05.485930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-9614e5fe-253e-4ee1-a746-439289d46f81 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:05.487200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9614e5fe-253e-4ee1-a746-439289d46f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:05.496730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75d825-1ac4-4f24-b096-526de3388901 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:05.578941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 134/275] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:05.642083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75d825-1ac4-4f24-b096-526de3388901 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:05.667337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614e5fe-253e-4ee1-a746-439289d46f81 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:05.680942 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cb00e-391e-4542-b00d-8e48db2a350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:05.681580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cb00e-391e-4542-b00d-8e48db2a350b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:05.681860 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cb00e-391e-4542-b00d-8e48db2a350b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:05.682135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cb00e-391e-4542-b00d-8e48db2a350b None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:05.696755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cb00e-391e-4542-b00d-8e48db2a350b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-666cb00e-391e-4542-b00d-8e48db2a350b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:05.703355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:05.704698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:05.705846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 142/276] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:05.713735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cb00e-391e-4542-b00d-8e48db2a350b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:05.748049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:05.794912 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614e5fe-253e-4ee1-a746-439289d46f81 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:05.816636 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:05.839644 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4a427-c74c-407b-94f3-6b5168febfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:05.840631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4a427-c74c-407b-94f3-6b5168febfff None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:05.840921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4a427-c74c-407b-94f3-6b5168febfff None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:05.841280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4a427-c74c-407b-94f3-6b5168febfff None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:05.842679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4a427-c74c-407b-94f3-6b5168febfff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:05.842880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cb00e-391e-4542-b00d-8e48db2a350b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:05.844700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4a427-c74c-407b-94f3-6b5168febfff None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:05.847202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 135/277] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:05.854534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4a427-c74c-407b-94f3-6b5168febfff None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:05.883403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db519952-36e3-4a0d-8c52-b6b506c29773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:05.884043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db519952-36e3-4a0d-8c52-b6b506c29773 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:05.884324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db519952-36e3-4a0d-8c52-b6b506c29773 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:05.884618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db519952-36e3-4a0d-8c52-b6b506c29773 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:05.885269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db519952-36e3-4a0d-8c52-b6b506c29773 None admin] RBAC: Adding query filter params (name=member) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:05.885570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db519952-36e3-4a0d-8c52-b6b506c29773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:05.887472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db519952-36e3-4a0d-8c52-b6b506c29773 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:05.895992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 143/278] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:05.900825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db519952-36e3-4a0d-8c52-b6b506c29773 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:05.980206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4a427-c74c-407b-94f3-6b5168febfff None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:06.007424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:06.031503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576ce44-d647-4c78-aad8-532265e2269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:06.044690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db519952-36e3-4a0d-8c52-b6b506c29773 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:06.044811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576ce44-d647-4c78-aad8-532265e2269a None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:06.044811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576ce44-d647-4c78-aad8-532265e2269a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:06.044811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576ce44-d647-4c78-aad8-532265e2269a None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:06.044811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576ce44-d647-4c78-aad8-532265e2269a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:06.044811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576ce44-d647-4c78-aad8-532265e2269a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:06.076439 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dbc56-395b-401e-b161-7c56339d7128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:06.077070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dbc56-395b-401e-b161-7c56339d7128 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:06.077348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dbc56-395b-401e-b161-7c56339d7128 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:06.077620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dbc56-395b-401e-b161-7c56339d7128 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:06.089297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dbc56-395b-401e-b161-7c56339d7128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-e20dbc56-395b-401e-b161-7c56339d7128 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:06.094078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:06.098769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 144/279] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:06.099769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dbc56-395b-401e-b161-7c56339d7128 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:06.148276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 136/280] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:31:05 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:31:06.149111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576ce44-d647-4c78-aad8-532265e2269a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:06.169282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:06.241706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dbc56-395b-401e-b161-7c56339d7128 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:06.266602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576ce44-d647-4c78-aad8-532265e2269a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:06.290864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14779e-a66b-494c-8fa3-bab960fc784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:06.292165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14779e-a66b-494c-8fa3-bab960fc784f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:06.292547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14779e-a66b-494c-8fa3-bab960fc784f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:06.292919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14779e-a66b-494c-8fa3-bab960fc784f None admin] PATH_INFO: `/v3/users/f78396199a8e4a229f9788ebd3d6cdc8` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:06.313884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14779e-a66b-494c-8fa3-bab960fc784f None admin] RBAC: Authorizing `identity:get_user(user_id=f78396199a8e4a229f9788ebd3d6cdc8)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:06.315564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df085a-4327-4cef-8a58-490e92fa663d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:06.315701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14779e-a66b-494c-8fa3-bab960fc784f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:06.316499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df085a-4327-4cef-8a58-490e92fa663d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:06.316783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df085a-4327-4cef-8a58-490e92fa663d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:06.317067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df085a-4327-4cef-8a58-490e92fa663d None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:06.317682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 145/281] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:06 2022] GET /identity/v3/users/f78396199a8e4a229f9788ebd3d6cdc8 => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:06.317806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df085a-4327-4cef-8a58-490e92fa663d None admin] RBAC: Adding query filter params (name=reader) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:06.318100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df085a-4327-4cef-8a58-490e92fa663d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:06.324176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df085a-4327-4cef-8a58-490e92fa663d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:06.325793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14779e-a66b-494c-8fa3-bab960fc784f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:06.330465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:06.339562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 137/282] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:06.345394 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df085a-4327-4cef-8a58-490e92fa663d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:06.484205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:06.489053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14779e-a66b-494c-8fa3-bab960fc784f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:06.514035 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df085a-4327-4cef-8a58-490e92fa663d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:06.538570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c96f59-e0e4-4bff-98bb-4cffd1ccea51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:06.539995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c96f59-e0e4-4bff-98bb-4cffd1ccea51 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:06.540584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c96f59-e0e4-4bff-98bb-4cffd1ccea51 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:06.540884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c96f59-e0e4-4bff-98bb-4cffd1ccea51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:06.541659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c96f59-e0e4-4bff-98bb-4cffd1ccea51 None admin] RBAC: Adding query filter params (role.id=0166494127b14615900b2c121908b2ba, user.id=f78396199a8e4a229f9788ebd3d6cdc8, scope.system=all) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:06.542029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c96f59-e0e4-4bff-98bb-4cffd1ccea51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:06.544603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c96f59-e0e4-4bff-98bb-4cffd1ccea51 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:06.552812 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 146/283] 10.222.0.98 () {56 vars in 1308 bytes} [Fri Aug 26 15:31:06 2022] GET /identity/v3/role_assignments?user.id=f78396199a8e4a229f9788ebd3d6cdc8&scope.system=all&role.id=0166494127b14615900b2c121908b2ba => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:06.555183 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a45cd-4dd1-4581-9517-49851b250223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:06.555841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a45cd-4dd1-4581-9517-49851b250223 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:06.556155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a45cd-4dd1-4581-9517-49851b250223 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:06.556434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a45cd-4dd1-4581-9517-49851b250223 None admin] PATH_INFO: `/v3/users/b486b19644f5496fa6370ea210bcef12` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:06.557650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a45cd-4dd1-4581-9517-49851b250223 None admin] RBAC: Authorizing `identity:get_user(user_id=b486b19644f5496fa6370ea210bcef12)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:06.559139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a45cd-4dd1-4581-9517-49851b250223 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:06.561140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 138/284] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:06 2022] GET /identity/v3/users/b486b19644f5496fa6370ea210bcef12 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:06.566750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c96f59-e0e4-4bff-98bb-4cffd1ccea51 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:06.670682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:06.720311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c96f59-e0e4-4bff-98bb-4cffd1ccea51 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:06.760356 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4df68-b6c7-4d01-ad26-fc5142efdbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:06.761042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4df68-b6c7-4d01-ad26-fc5142efdbdc None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:06.761330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4df68-b6c7-4d01-ad26-fc5142efdbdc None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:06.761610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4df68-b6c7-4d01-ad26-fc5142efdbdc None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:06.762800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4df68-b6c7-4d01-ad26-fc5142efdbdc None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:06.764658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4df68-b6c7-4d01-ad26-fc5142efdbdc None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:06.766533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 147/285] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:06 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:06.773725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a45cd-4dd1-4581-9517-49851b250223 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:06.898559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a45cd-4dd1-4581-9517-49851b250223 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:06.934978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1aa3e5-94ae-4b67-bad5-65bd6019d611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:06.935649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1aa3e5-94ae-4b67-bad5-65bd6019d611 None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:06.935960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1aa3e5-94ae-4b67-bad5-65bd6019d611 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:06.936238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1aa3e5-94ae-4b67-bad5-65bd6019d611 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280/users/b486b19644f5496fa6370ea210bcef12/roles/1c9432f222d8471bb13d0479297cf5d5` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:06.938171 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1aa3e5-94ae-4b67-bad5-65bd6019d611 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee18025f6d34d0daf407c44c5f28280, user_id=b486b19644f5496fa6370ea210bcef12, role_id=1c9432f222d8471bb13d0479297cf5d5)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:06.939608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1aa3e5-94ae-4b67-bad5-65bd6019d611 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:07.013933 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 139/286] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:31:06 2022] PUT /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280/users/b486b19644f5496fa6370ea210bcef12/roles/1c9432f222d8471bb13d0479297cf5d5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:31:07.415067 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5122d3-6ae9-4c7b-8835-9307d8787786 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:07.415584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5122d3-6ae9-4c7b-8835-9307d8787786 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:07.415660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5122d3-6ae9-4c7b-8835-9307d8787786 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:07.417353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 148/287] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:07.423102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b21c4e-de1e-4712-9823-679c850e5ac1 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:07.427296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b21c4e-de1e-4712-9823-679c850e5ac1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:07.427296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b21c4e-de1e-4712-9823-679c850e5ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:07.433509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:07.444994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-14b21c4e-de1e-4712-9823-679c850e5ac1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:07.450280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-14b21c4e-de1e-4712-9823-679c850e5ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:07.526994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 140/288] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:07.893472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5122d3-6ae9-4c7b-8835-9307d8787786 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:08.020561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5122d3-6ae9-4c7b-8835-9307d8787786 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:08.066842 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2e070-6fbd-4668-b055-526dbfd13fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:08.067471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2e070-6fbd-4668-b055-526dbfd13fbd None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:08.067755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2e070-6fbd-4668-b055-526dbfd13fbd None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:08.068065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2e070-6fbd-4668-b055-526dbfd13fbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:08.070879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:08.081367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2e070-6fbd-4668-b055-526dbfd13fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-cfe2e070-6fbd-4668-b055-526dbfd13fbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:08.086161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:08.090681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:08.091488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 149/289] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:08.093708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b21c4e-de1e-4712-9823-679c850e5ac1 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:08.266340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b21c4e-de1e-4712-9823-679c850e5ac1 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:08.302779 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d621b8-74a7-431d-9738-eba4a653f7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:08.303397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d621b8-74a7-431d-9738-eba4a653f7c6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:08.303708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d621b8-74a7-431d-9738-eba4a653f7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:08.304021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d621b8-74a7-431d-9738-eba4a653f7c6 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:08.304696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d621b8-74a7-431d-9738-eba4a653f7c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:08.305001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d621b8-74a7-431d-9738-eba4a653f7c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:08.306369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d621b8-74a7-431d-9738-eba4a653f7c6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:08.307800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:08.315475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 141/290] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:08.320961 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2e070-6fbd-4668-b055-526dbfd13fbd None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:08.428838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2e070-6fbd-4668-b055-526dbfd13fbd None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:08.477146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2caf0-8be9-4190-9d75-02188b34fa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:08.477791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2caf0-8be9-4190-9d75-02188b34fa50 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:08.478076 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2caf0-8be9-4190-9d75-02188b34fa50 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:08.481941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2caf0-8be9-4190-9d75-02188b34fa50 None admin] PATH_INFO: `/v3/users/a8d5924fb50340998b79d9061e2bfc28` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:08.481941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2caf0-8be9-4190-9d75-02188b34fa50 None admin] RBAC: Authorizing `identity:get_user(user_id=a8d5924fb50340998b79d9061e2bfc28)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:08.481941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2caf0-8be9-4190-9d75-02188b34fa50 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:08.482563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 150/291] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:08 2022] GET /identity/v3/users/a8d5924fb50340998b79d9061e2bfc28 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:08.488271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d621b8-74a7-431d-9738-eba4a653f7c6 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:08.601143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d621b8-74a7-431d-9738-eba4a653f7c6 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:08.635925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb23bd5-b87d-401e-8638-ee5de9e04d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:08.636658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb23bd5-b87d-401e-8638-ee5de9e04d86 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:08.636941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb23bd5-b87d-401e-8638-ee5de9e04d86 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:08.637220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb23bd5-b87d-401e-8638-ee5de9e04d86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:08.708236 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:08.712239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb23bd5-b87d-401e-8638-ee5de9e04d86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:08.713700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb23bd5-b87d-401e-8638-ee5de9e04d86 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:08.786885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00787ab-fc7f-4d95-8cab-639be2279de4 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:08.787206 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00787ab-fc7f-4d95-8cab-639be2279de4 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:08.787442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00787ab-fc7f-4d95-8cab-639be2279de4 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:08.789039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 151/292] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:08.794510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1023c8-af49-47e7-8cc0-a481f4715820 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:08.794791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1023c8-af49-47e7-8cc0-a481f4715820 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:08.795069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1023c8-af49-47e7-8cc0-a481f4715820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:08.825441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-af1023c8-af49-47e7-8cc0-a481f4715820 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:08.826640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-af1023c8-af49-47e7-8cc0-a481f4715820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:08.862410 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 142/293] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:08.871183 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb23bd5-b87d-401e-8638-ee5de9e04d86 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:08.937564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:08.951904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 152/294] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 26 15:31:08.951904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f628b-335f-447a-a58c-e46c41f81a9e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:08.951904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f628b-335f-447a-a58c-e46c41f81a9e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:08.951904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f628b-335f-447a-a58c-e46c41f81a9e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:08.953747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 153/295] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:08.953747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1da7af-7d9e-4449-b877-686a6524eea8 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:08.953747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1da7af-7d9e-4449-b877-686a6524eea8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:08.954042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1da7af-7d9e-4449-b877-686a6524eea8 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:08.955483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 154/296] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:08.958509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cec5be-f3d9-42c1-880f-b62d955a5662 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:08.958797 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cec5be-f3d9-42c1-880f-b62d955a5662 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:08.959132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cec5be-f3d9-42c1-880f-b62d955a5662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:08.979959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-b5cec5be-f3d9-42c1-880f-b62d955a5662 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:08.981170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-b5cec5be-f3d9-42c1-880f-b62d955a5662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:08.985673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:08.992425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb23bd5-b87d-401e-8638-ee5de9e04d86 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:09.028327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0e5db-25da-44c4-90d2-17d1b9c938cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:09.029011 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e5db-25da-44c4-90d2-17d1b9c938cf None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:09.029298 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e5db-25da-44c4-90d2-17d1b9c938cf None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:09.029578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e5db-25da-44c4-90d2-17d1b9c938cf None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:09.030756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0e5db-25da-44c4-90d2-17d1b9c938cf None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:09.032270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0e5db-25da-44c4-90d2-17d1b9c938cf None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:09.034341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 143/297] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:08 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:09.035534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cee61-1558-4d39-a8fb-49e2a5339716 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:09.035825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cee61-1558-4d39-a8fb-49e2a5339716 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:09.036129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cee61-1558-4d39-a8fb-49e2a5339716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:09.051440 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 155/298] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:09.052682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cec5be-f3d9-42c1-880f-b62d955a5662 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:09.062261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-8d8cee61-1558-4d39-a8fb-49e2a5339716 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:09.063567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8d8cee61-1558-4d39-a8fb-49e2a5339716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:09.169105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cec5be-f3d9-42c1-880f-b62d955a5662 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:09.196777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:09.213761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f25764-4ef8-4c81-8ab5-cf42d4ff4761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:09.215861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f25764-4ef8-4c81-8ab5-cf42d4ff4761 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:09.217678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f25764-4ef8-4c81-8ab5-cf42d4ff4761 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:09.218142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f25764-4ef8-4c81-8ab5-cf42d4ff4761 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:09.219056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f25764-4ef8-4c81-8ab5-cf42d4ff4761 None admin] RBAC: Adding query filter params (role.id=0166494127b14615900b2c121908b2ba, scope.project.id=1ee18025f6d34d0daf407c44c5f28280, user.id=a8d5924fb50340998b79d9061e2bfc28) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:09.220440 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f25764-4ef8-4c81-8ab5-cf42d4ff4761 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:09.223181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f25764-4ef8-4c81-8ab5-cf42d4ff4761 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:09.231254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 144/299] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:09.234261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 156/300] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:09 2022] GET /identity/v3/role_assignments?user.id=a8d5924fb50340998b79d9061e2bfc28&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=0166494127b14615900b2c121908b2ba => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:09.303021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cee61-1558-4d39-a8fb-49e2a5339716 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:09.389076 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f25764-4ef8-4c81-8ab5-cf42d4ff4761 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:09.431938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:09.440547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cee61-1558-4d39-a8fb-49e2a5339716 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:09.482292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd4ac3-9e07-4611-85da-c92f9a6b366f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:09.483051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd4ac3-9e07-4611-85da-c92f9a6b366f None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:09.483404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd4ac3-9e07-4611-85da-c92f9a6b366f None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:09.483797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd4ac3-9e07-4611-85da-c92f9a6b366f None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:09.485297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd4ac3-9e07-4611-85da-c92f9a6b366f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:09.486821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd4ac3-9e07-4611-85da-c92f9a6b366f None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:09.488847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 145/301] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:09.493963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd4ac3-9e07-4611-85da-c92f9a6b366f None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:09.543834 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f25764-4ef8-4c81-8ab5-cf42d4ff4761 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:09.596296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f3c4a-3d89-4981-ab9b-bde8c180a127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:09.596944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f3c4a-3d89-4981-ab9b-bde8c180a127 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:09.597230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f3c4a-3d89-4981-ab9b-bde8c180a127 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:09.597513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f3c4a-3d89-4981-ab9b-bde8c180a127 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:09.613691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f3c4a-3d89-4981-ab9b-bde8c180a127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-625f3c4a-3d89-4981-ab9b-bde8c180a127 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:09.619958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:09.620869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 157/302] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:09.622579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f3c4a-3d89-4981-ab9b-bde8c180a127 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:09.642324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd4ac3-9e07-4611-85da-c92f9a6b366f None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:09.680731 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8533184-ae54-4e3f-92f9-589489c44f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:09.681367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8533184-ae54-4e3f-92f9-589489c44f65 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:09.681652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8533184-ae54-4e3f-92f9-589489c44f65 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:09.682013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8533184-ae54-4e3f-92f9-589489c44f65 None admin] PATH_INFO: `/v3/groups` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:09.682576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8533184-ae54-4e3f-92f9-589489c44f65 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:09.684044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8533184-ae54-4e3f-92f9-589489c44f65 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:09.684442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:09.737793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f3c4a-3d89-4981-ab9b-bde8c180a127 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:09.765402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 146/303] 10.222.0.98 () {60 vars in 1134 bytes} [Fri Aug 26 15:31:09 2022] POST /identity/v3/groups => generated 229 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:31:09.766209 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8533184-ae54-4e3f-92f9-589489c44f65 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:09.782718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eee78-942c-44b0-b7a6-fa3dec32beeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:09.784077 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eee78-942c-44b0-b7a6-fa3dec32beeb None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:09.784552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eee78-942c-44b0-b7a6-fa3dec32beeb None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:09.785378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eee78-942c-44b0-b7a6-fa3dec32beeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:09.798400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:09.807013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081eee78-942c-44b0-b7a6-fa3dec32beeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-081eee78-942c-44b0-b7a6-fa3dec32beeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:09.812675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:09.813713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:09.814596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:09.819059 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 158/304] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:09.820164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eee78-942c-44b0-b7a6-fa3dec32beeb None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:09.903839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8533184-ae54-4e3f-92f9-589489c44f65 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:09.945814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eee78-942c-44b0-b7a6-fa3dec32beeb None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:09.947716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacb997-b44b-464d-a242-2160656632c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:09.948874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacb997-b44b-464d-a242-2160656632c7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:09.948874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacb997-b44b-464d-a242-2160656632c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:09.949064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacb997-b44b-464d-a242-2160656632c7 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:09.949587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacb997-b44b-464d-a242-2160656632c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:09.949892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacb997-b44b-464d-a242-2160656632c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:09.951286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacb997-b44b-464d-a242-2160656632c7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:09.955645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:09.961600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 147/305] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:09.962763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a8a1-30cb-44c6-8582-1d9bf86ea8b4 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:09.963150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a8a1-30cb-44c6-8582-1d9bf86ea8b4 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:09.963331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a8a1-30cb-44c6-8582-1d9bf86ea8b4 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:09.964665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 148/306] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:09.969438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936a8a1-30cb-44c6-8582-1d9bf86ea8b4 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:09.986357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bde05-a921-4c47-b132-82c06921f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:09.987038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bde05-a921-4c47-b132-82c06921f9fa None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:09.987331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bde05-a921-4c47-b132-82c06921f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:09.987612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bde05-a921-4c47-b132-82c06921f9fa None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:09.988413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bde05-a921-4c47-b132-82c06921f9fa None admin] RBAC: Adding query filter params (name=member) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:09.988987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bde05-a921-4c47-b132-82c06921f9fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:09.991116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bde05-a921-4c47-b132-82c06921f9fa None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:09.999375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 159/307] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:10.000557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc45835-ac32-40b5-9cc7-5e917b862d2b None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:10.000842 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc45835-ac32-40b5-9cc7-5e917b862d2b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:10.001118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc45835-ac32-40b5-9cc7-5e917b862d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:10.018907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-6bc45835-ac32-40b5-9cc7-5e917b862d2b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:10.020135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6bc45835-ac32-40b5-9cc7-5e917b862d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:10.074116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:10.092277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936a8a1-30cb-44c6-8582-1d9bf86ea8b4 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:10.094031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 160/308] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:10.094693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc45835-ac32-40b5-9cc7-5e917b862d2b None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:10.139904 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d86770-5642-40f4-8021-1e8a1de1938b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:10.140866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d86770-5642-40f4-8021-1e8a1de1938b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:10.141298 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d86770-5642-40f4-8021-1e8a1de1938b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:10.141762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d86770-5642-40f4-8021-1e8a1de1938b None admin] PATH_INFO: `/v3/users/32c3302597ab4cd38c4550fda4d6dfad` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:10.157262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d86770-5642-40f4-8021-1e8a1de1938b None admin] RBAC: Authorizing `identity:get_user(user_id=32c3302597ab4cd38c4550fda4d6dfad)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:10.159773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d86770-5642-40f4-8021-1e8a1de1938b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:10.168062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 149/309] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:09 2022] GET /identity/v3/users/32c3302597ab4cd38c4550fda4d6dfad => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:10.169582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d86770-5642-40f4-8021-1e8a1de1938b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:10.212717 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:10.215540 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc45835-ac32-40b5-9cc7-5e917b862d2b None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:10.254171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad44bab-1b50-42fc-897f-16460c8c3186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:10.254972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad44bab-1b50-42fc-897f-16460c8c3186 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:10.255352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad44bab-1b50-42fc-897f-16460c8c3186 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:10.255726 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad44bab-1b50-42fc-897f-16460c8c3186 None admin] PATH_INFO: `/v3/users/f78396199a8e4a229f9788ebd3d6cdc8` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:10.257126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad44bab-1b50-42fc-897f-16460c8c3186 None admin] RBAC: Authorizing `identity:get_user(user_id=f78396199a8e4a229f9788ebd3d6cdc8)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:10.258879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad44bab-1b50-42fc-897f-16460c8c3186 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:10.265093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 161/310] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:10 2022] GET /identity/v3/users/f78396199a8e4a229f9788ebd3d6cdc8 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:10.266059 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad44bab-1b50-42fc-897f-16460c8c3186 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:10.310154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d86770-5642-40f4-8021-1e8a1de1938b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:10.357739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402324e0-94fe-4d99-b5e0-7871b032b161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:10.358964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402324e0-94fe-4d99-b5e0-7871b032b161 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:10.359909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402324e0-94fe-4d99-b5e0-7871b032b161 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:10.360397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402324e0-94fe-4d99-b5e0-7871b032b161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:10.361605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402324e0-94fe-4d99-b5e0-7871b032b161 None admin] RBAC: Adding query filter params (role.id=1c9432f222d8471bb13d0479297cf5d5, user.id=32c3302597ab4cd38c4550fda4d6dfad, scope.system=all) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:10.362380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402324e0-94fe-4d99-b5e0-7871b032b161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:10.364481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402324e0-94fe-4d99-b5e0-7871b032b161 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:10.380500 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 150/311] 10.222.0.98 () {56 vars in 1308 bytes} [Fri Aug 26 15:31:10 2022] GET /identity/v3/role_assignments?user.id=32c3302597ab4cd38c4550fda4d6dfad&scope.system=all&role.id=1c9432f222d8471bb13d0479297cf5d5 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:10.388854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad44bab-1b50-42fc-897f-16460c8c3186 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:10.406703 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402324e0-94fe-4d99-b5e0-7871b032b161 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:10.415853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:10.436792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbef2d6-3e4b-4dac-a2bf-0258c1601800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:10.438103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbef2d6-3e4b-4dac-a2bf-0258c1601800 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:10.438395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbef2d6-3e4b-4dac-a2bf-0258c1601800 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:10.438677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbef2d6-3e4b-4dac-a2bf-0258c1601800 None admin] PATH_INFO: `/v3/system/users/f78396199a8e4a229f9788ebd3d6cdc8/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:10.441224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbef2d6-3e4b-4dac-a2bf-0258c1601800 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f78396199a8e4a229f9788ebd3d6cdc8, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:10.442788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbef2d6-3e4b-4dac-a2bf-0258c1601800 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:10.521377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402324e0-94fe-4d99-b5e0-7871b032b161 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:10.522272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 162/312] 10.222.0.98 () {58 vars in 1334 bytes} [Fri Aug 26 15:31:10 2022] PUT /identity/v3/system/users/f78396199a8e4a229f9788ebd3d6cdc8/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:31:10.582624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecda0c-031e-485f-8f96-37108e8d294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:10.583301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecda0c-031e-485f-8f96-37108e8d294f None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:10.583586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecda0c-031e-485f-8f96-37108e8d294f None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:10.583884 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecda0c-031e-485f-8f96-37108e8d294f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:10.598071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecda0c-031e-485f-8f96-37108e8d294f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-deecda0c-031e-485f-8f96-37108e8d294f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:10.603985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:10.605847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:10.606675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:10.610272 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 151/313] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:10.611700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbef2d6-3e4b-4dac-a2bf-0258c1601800 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:10.652308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:10.726546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbef2d6-3e4b-4dac-a2bf-0258c1601800 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:10.764575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e90285-88f9-4687-8863-24978889fa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:10.765187 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e90285-88f9-4687-8863-24978889fa79 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:10.765471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e90285-88f9-4687-8863-24978889fa79 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:10.765749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e90285-88f9-4687-8863-24978889fa79 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:10.766424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e90285-88f9-4687-8863-24978889fa79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:10.766729 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e90285-88f9-4687-8863-24978889fa79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:10.768148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e90285-88f9-4687-8863-24978889fa79 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:10.777243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 163/314] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:10.782642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecda0c-031e-485f-8f96-37108e8d294f None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:10.853375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:10.893712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecda0c-031e-485f-8f96-37108e8d294f None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:10.937109 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761cae9-02e9-4414-9b14-8cb245ca4ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:10.937745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761cae9-02e9-4414-9b14-8cb245ca4ed3 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:10.938029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761cae9-02e9-4414-9b14-8cb245ca4ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:10.938308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761cae9-02e9-4414-9b14-8cb245ca4ed3 None admin] PATH_INFO: `/v3/users/b486b19644f5496fa6370ea210bcef12` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:10.939445 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761cae9-02e9-4414-9b14-8cb245ca4ed3 None admin] RBAC: Authorizing `identity:get_user(user_id=b486b19644f5496fa6370ea210bcef12)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:10.940925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761cae9-02e9-4414-9b14-8cb245ca4ed3 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:10.942498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 152/315] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:10 2022] GET /identity/v3/users/b486b19644f5496fa6370ea210bcef12 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:10.953009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e90285-88f9-4687-8863-24978889fa79 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:11.052431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:11.059611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e90285-88f9-4687-8863-24978889fa79 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:11.095072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe14f8-a7d0-47a7-8efc-4354c479e979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:11.095743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe14f8-a7d0-47a7-8efc-4354c479e979 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:11.096065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe14f8-a7d0-47a7-8efc-4354c479e979 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:11.096347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe14f8-a7d0-47a7-8efc-4354c479e979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:11.166919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe14f8-a7d0-47a7-8efc-4354c479e979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:11.168441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe14f8-a7d0-47a7-8efc-4354c479e979 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:11.272241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:11.275952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 164/316] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:11.277314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761cae9-02e9-4414-9b14-8cb245ca4ed3 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:11.385986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761cae9-02e9-4414-9b14-8cb245ca4ed3 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:11.422815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefd56f-7f2b-419c-8e30-0a1147436120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:11.423451 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefd56f-7f2b-419c-8e30-0a1147436120 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:11.423729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefd56f-7f2b-419c-8e30-0a1147436120 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:11.430174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefd56f-7f2b-419c-8e30-0a1147436120 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:11.430174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefd56f-7f2b-419c-8e30-0a1147436120 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:11.430174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefd56f-7f2b-419c-8e30-0a1147436120 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:11.430174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 153/317] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:11 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:11.434688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe14f8-a7d0-47a7-8efc-4354c479e979 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:11.543160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe14f8-a7d0-47a7-8efc-4354c479e979 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:11.580764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f23f5f-fce6-4332-a5a8-39640d0cc97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:11.581383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23f5f-fce6-4332-a5a8-39640d0cc97a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:11.581719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23f5f-fce6-4332-a5a8-39640d0cc97a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:11.582005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23f5f-fce6-4332-a5a8-39640d0cc97a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:11.582740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f23f5f-fce6-4332-a5a8-39640d0cc97a None admin] RBAC: Adding query filter params (role.id=1c9432f222d8471bb13d0479297cf5d5, scope.project.id=1ee18025f6d34d0daf407c44c5f28280, user.id=b486b19644f5496fa6370ea210bcef12) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:11.583054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f23f5f-fce6-4332-a5a8-39640d0cc97a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:11.584533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f23f5f-fce6-4332-a5a8-39640d0cc97a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:11.596087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 165/318] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:11 2022] GET /identity/v3/role_assignments?user.id=b486b19644f5496fa6370ea210bcef12&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=1c9432f222d8471bb13d0479297cf5d5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:11.780377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2f1e-4bdd-4b3d-8cbb-8794e211bae8 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:11.780670 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2f1e-4bdd-4b3d-8cbb-8794e211bae8 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:11.780959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2f1e-4bdd-4b3d-8cbb-8794e211bae8 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:11.782842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 154/319] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:11.788073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565359c5-9e1e-4fb5-b90a-966af6257d02 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:11.788965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565359c5-9e1e-4fb5-b90a-966af6257d02 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:11.789458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565359c5-9e1e-4fb5-b90a-966af6257d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:11.809249 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-565359c5-9e1e-4fb5-b90a-966af6257d02 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:11.810930 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-565359c5-9e1e-4fb5-b90a-966af6257d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:11.869532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:11.883441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 166/320] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:12.145738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d2f1e-4bdd-4b3d-8cbb-8794e211bae8 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:12.148610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:12.255027 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d2f1e-4bdd-4b3d-8cbb-8794e211bae8 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:12.289590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f893a-2e26-4978-99b5-29934b3e8f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:12.290205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f893a-2e26-4978-99b5-29934b3e8f41 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:12.290482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f893a-2e26-4978-99b5-29934b3e8f41 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:12.290750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f893a-2e26-4978-99b5-29934b3e8f41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:12.291829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f893a-2e26-4978-99b5-29934b3e8f41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:12.293310 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f893a-2e26-4978-99b5-29934b3e8f41 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:12.295084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 155/321] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:12.300353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565359c5-9e1e-4fb5-b90a-966af6257d02 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:12.402061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565359c5-9e1e-4fb5-b90a-966af6257d02 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:12.408175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2c789-d4f0-435c-bc16-f5c0d5eb1655 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:12.408618 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2c789-d4f0-435c-bc16-f5c0d5eb1655 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:12.408787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2c789-d4f0-435c-bc16-f5c0d5eb1655 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:12.410268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 156/322] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:12.415542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535def7-2f71-4827-85d2-8705fc343365 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:12.415839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535def7-2f71-4827-85d2-8705fc343365 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:12.416151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535def7-2f71-4827-85d2-8705fc343365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:12.439995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9e4f1-de6f-4d42-af94-5a5defb1da5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:12.440202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7535def7-2f71-4827-85d2-8705fc343365 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:12.440616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e4f1-de6f-4d42-af94-5a5defb1da5e None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:12.440902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e4f1-de6f-4d42-af94-5a5defb1da5e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:12.441179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e4f1-de6f-4d42-af94-5a5defb1da5e None admin] PATH_INFO: `/v3/groups` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:12.441652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7535def7-2f71-4827-85d2-8705fc343365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:12.441821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9e4f1-de6f-4d42-af94-5a5defb1da5e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:12.443293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9e4f1-de6f-4d42-af94-5a5defb1da5e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:12.561841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 167/323] 10.222.0.98 () {60 vars in 1134 bytes} [Fri Aug 26 15:31:12 2022] POST /identity/v3/groups => generated 226 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:31:12.570617 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f688b3-b227-4ec2-9549-3711d574e307 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:12.570944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f688b3-b227-4ec2-9549-3711d574e307 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:12.571258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f688b3-b227-4ec2-9549-3711d574e307 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:12.572729 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 168/324] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:12.579128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e089c3-7df0-40a0-9b43-f75a7ceb6e73 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:12.579128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e089c3-7df0-40a0-9b43-f75a7ceb6e73 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:12.579401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e089c3-7df0-40a0-9b43-f75a7ceb6e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:12.591444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:12.599968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 157/325] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:12.605340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e0e089c3-7df0-40a0-9b43-f75a7ceb6e73 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:12.607281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e0e089c3-7df0-40a0-9b43-f75a7ceb6e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:12.672110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:12.697896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 169/326] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:12.894050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535def7-2f71-4827-85d2-8705fc343365 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:12.976631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e089c3-7df0-40a0-9b43-f75a7ceb6e73 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:13.008479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535def7-2f71-4827-85d2-8705fc343365 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:13.062884 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d971e-0814-4655-bf3a-d20e6f054328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:13.064160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d971e-0814-4655-bf3a-d20e6f054328 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:13.065051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d971e-0814-4655-bf3a-d20e6f054328 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:13.065855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d971e-0814-4655-bf3a-d20e6f054328 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:13.081540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d971e-0814-4655-bf3a-d20e6f054328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-323d971e-0814-4655-bf3a-d20e6f054328 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:13.088285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:13.089224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:13.090032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:13.092619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 158/327] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:13.099450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d971e-0814-4655-bf3a-d20e6f054328 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:13.120289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e089c3-7df0-40a0-9b43-f75a7ceb6e73 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:13.180708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0e18b-dba4-49e0-9db6-147e57470b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:13.181364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0e18b-dba4-49e0-9db6-147e57470b54 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:13.181650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0e18b-dba4-49e0-9db6-147e57470b54 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:13.181928 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0e18b-dba4-49e0-9db6-147e57470b54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:13.200964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:13.202598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0e18b-dba4-49e0-9db6-147e57470b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-cfa0e18b-dba4-49e0-9db6-147e57470b54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:13.208874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:13.209783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:13.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:13.211249 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 170/328] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:13.217861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0e18b-dba4-49e0-9db6-147e57470b54 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:13.256893 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d971e-0814-4655-bf3a-d20e6f054328 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:13.289202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:13.305753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4051b-3e47-401c-911a-3f0e486f6a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:13.306379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4051b-3e47-401c-911a-3f0e486f6a2d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:13.306662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4051b-3e47-401c-911a-3f0e486f6a2d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:13.306959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4051b-3e47-401c-911a-3f0e486f6a2d None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:13.307841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4051b-3e47-401c-911a-3f0e486f6a2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:13.307969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4051b-3e47-401c-911a-3f0e486f6a2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:13.309384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4051b-3e47-401c-911a-3f0e486f6a2d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:13.318807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 159/329] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:13.324222 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4051b-3e47-401c-911a-3f0e486f6a2d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:13.393658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0e18b-dba4-49e0-9db6-147e57470b54 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:13.438051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f51c8-649e-4154-bf02-a37170af5185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:13.438664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f51c8-649e-4154-bf02-a37170af5185 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:13.438947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f51c8-649e-4154-bf02-a37170af5185 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:13.439221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f51c8-649e-4154-bf02-a37170af5185 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:13.439923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f51c8-649e-4154-bf02-a37170af5185 None admin] RBAC: Adding query filter params (name=member) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:13.440237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f51c8-649e-4154-bf02-a37170af5185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:13.441710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f51c8-649e-4154-bf02-a37170af5185 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:13.453099 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 171/330] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:13.458628 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f51c8-649e-4154-bf02-a37170af5185 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:13.465201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4051b-3e47-401c-911a-3f0e486f6a2d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:13.493396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:13.508088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002897ff-b8f8-4d8c-80f4-1cf945cf35da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:13.508654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002897ff-b8f8-4d8c-80f4-1cf945cf35da None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:13.509428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002897ff-b8f8-4d8c-80f4-1cf945cf35da None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:13.509709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002897ff-b8f8-4d8c-80f4-1cf945cf35da None admin] PATH_INFO: `/v3/users/32c3302597ab4cd38c4550fda4d6dfad` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:13.510880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002897ff-b8f8-4d8c-80f4-1cf945cf35da None admin] RBAC: Authorizing `identity:get_user(user_id=32c3302597ab4cd38c4550fda4d6dfad)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:13.512759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002897ff-b8f8-4d8c-80f4-1cf945cf35da None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:13.515185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 160/331] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:13 2022] GET /identity/v3/users/32c3302597ab4cd38c4550fda4d6dfad => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:13.520610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002897ff-b8f8-4d8c-80f4-1cf945cf35da None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:13.577253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:13.580265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f51c8-649e-4154-bf02-a37170af5185 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:13.618887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b5a18-3728-41ec-a6eb-9835df0bc198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:13.619804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b5a18-3728-41ec-a6eb-9835df0bc198 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:13.620256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b5a18-3728-41ec-a6eb-9835df0bc198 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:13.620672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b5a18-3728-41ec-a6eb-9835df0bc198 None admin] PATH_INFO: `/v3/users/f78396199a8e4a229f9788ebd3d6cdc8` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:13.621992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b5a18-3728-41ec-a6eb-9835df0bc198 None admin] RBAC: Authorizing `identity:get_user(user_id=f78396199a8e4a229f9788ebd3d6cdc8)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:13.623613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b5a18-3728-41ec-a6eb-9835df0bc198 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:13.625779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 172/332] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:13 2022] GET /identity/v3/users/f78396199a8e4a229f9788ebd3d6cdc8 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:13.631695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b5a18-3728-41ec-a6eb-9835df0bc198 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:13.637095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002897ff-b8f8-4d8c-80f4-1cf945cf35da None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:13.674885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f48cde-5ed5-469e-b516-82aef3c89cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:13.675549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f48cde-5ed5-469e-b516-82aef3c89cb4 None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:13.675834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f48cde-5ed5-469e-b516-82aef3c89cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:13.676163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f48cde-5ed5-469e-b516-82aef3c89cb4 None admin] PATH_INFO: `/v3/system/users/32c3302597ab4cd38c4550fda4d6dfad/roles/1c9432f222d8471bb13d0479297cf5d5` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:13.677848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f48cde-5ed5-469e-b516-82aef3c89cb4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=32c3302597ab4cd38c4550fda4d6dfad, role_id=1c9432f222d8471bb13d0479297cf5d5)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:13.679892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f48cde-5ed5-469e-b516-82aef3c89cb4 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:13.747364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b5a18-3728-41ec-a6eb-9835df0bc198 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:13.765989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 161/333] 10.222.0.98 () {58 vars in 1334 bytes} [Fri Aug 26 15:31:13 2022] PUT /identity/v3/system/users/32c3302597ab4cd38c4550fda4d6dfad/roles/1c9432f222d8471bb13d0479297cf5d5 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:31:13.785325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4651f2-bf8a-438a-b7f0-7935bb90664b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:13.786247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4651f2-bf8a-438a-b7f0-7935bb90664b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:13.786722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4651f2-bf8a-438a-b7f0-7935bb90664b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:13.795756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4651f2-bf8a-438a-b7f0-7935bb90664b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:13.795756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4651f2-bf8a-438a-b7f0-7935bb90664b None admin] RBAC: Adding query filter params (role.id=0166494127b14615900b2c121908b2ba, user.id=f78396199a8e4a229f9788ebd3d6cdc8, scope.system=all) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:13.795756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4651f2-bf8a-438a-b7f0-7935bb90664b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:13.795756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4651f2-bf8a-438a-b7f0-7935bb90664b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:13.805496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 173/334] 10.222.0.98 () {56 vars in 1308 bytes} [Fri Aug 26 15:31:13 2022] GET /identity/v3/role_assignments?user.id=f78396199a8e4a229f9788ebd3d6cdc8&scope.system=all&role.id=0166494127b14615900b2c121908b2ba => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:15.032557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445979bb-edc6-421f-b280-bed35d31ed14 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:15.033093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445979bb-edc6-421f-b280-bed35d31ed14 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:15.033426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445979bb-edc6-421f-b280-bed35d31ed14 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:15.035713 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 162/335] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:15.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8e629-5085-4eab-b233-2823f428f48f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:15.041931 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8e629-5085-4eab-b233-2823f428f48f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:15.042411 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8e629-5085-4eab-b233-2823f428f48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:15.066725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-50f8e629-5085-4eab-b233-2823f428f48f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:15.068358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-50f8e629-5085-4eab-b233-2823f428f48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:15.091320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:15.154567 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 174/336] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:15.477469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c188b-98e1-49c2-a085-c4da5b93465a None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:15.477842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c188b-98e1-49c2-a085-c4da5b93465a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:15.478020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c188b-98e1-49c2-a085-c4da5b93465a None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:15.479611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 163/337] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:15.485731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9acb45-2278-4094-8aa7-932e5e0cb65b None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:15.486031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9acb45-2278-4094-8aa7-932e5e0cb65b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:15.486308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9acb45-2278-4094-8aa7-932e5e0cb65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:15.506870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3f9acb45-2278-4094-8aa7-932e5e0cb65b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:15.508131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3f9acb45-2278-4094-8aa7-932e5e0cb65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:15.583996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 175/338] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:15.715438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c188b-98e1-49c2-a085-c4da5b93465a None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:15.779138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:15.826131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c188b-98e1-49c2-a085-c4da5b93465a None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:15.869359 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6ab12-a059-42ca-be57-8edfa02e1a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:15.869998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6ab12-a059-42ca-be57-8edfa02e1a23 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:15.870283 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6ab12-a059-42ca-be57-8edfa02e1a23 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:15.870561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6ab12-a059-42ca-be57-8edfa02e1a23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:15.886049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6ab12-a059-42ca-be57-8edfa02e1a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-b6d6ab12-a059-42ca-be57-8edfa02e1a23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:15.890750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:15.891635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:15.892484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:15.896223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 164/339] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:15.899735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9acb45-2278-4094-8aa7-932e5e0cb65b None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:15.940469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6ab12-a059-42ca-be57-8edfa02e1a23 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:16.100513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:16.134081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9acb45-2278-4094-8aa7-932e5e0cb65b None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:16.138652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6ab12-a059-42ca-be57-8edfa02e1a23 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:16.168248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:16.209615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091a64-5319-4601-94fb-a4175c55fce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:16.210264 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091a64-5319-4601-94fb-a4175c55fce4 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:16.210573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091a64-5319-4601-94fb-a4175c55fce4 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:16.210846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091a64-5319-4601-94fb-a4175c55fce4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:16.229592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28091a64-5319-4601-94fb-a4175c55fce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:16.231479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591c48c-8bc5-4083-b88e-4976e4bab569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:16.232143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591c48c-8bc5-4083-b88e-4976e4bab569 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:16.232430 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591c48c-8bc5-4083-b88e-4976e4bab569 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:16.232709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591c48c-8bc5-4083-b88e-4976e4bab569 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:16.233389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591c48c-8bc5-4083-b88e-4976e4bab569 None admin] RBAC: Adding query filter params (name=member) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:16.233755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591c48c-8bc5-4083-b88e-4976e4bab569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-28091a64-5319-4601-94fb-a4175c55fce4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:16.234334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:16.235212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:16.236053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:16.236678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591c48c-8bc5-4083-b88e-4976e4bab569 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:16.244263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 165/340] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:31:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:16.245204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091a64-5319-4601-94fb-a4175c55fce4 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:16.262147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 176/341] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:16.272862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591c48c-8bc5-4083-b88e-4976e4bab569 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:16.389896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091a64-5319-4601-94fb-a4175c55fce4 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:16.437416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9e416-b6be-4d11-83e2-f468a5c0bdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:16.438043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9e416-b6be-4d11-83e2-f468a5c0bdca None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:16.438324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9e416-b6be-4d11-83e2-f468a5c0bdca None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:16.438597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9e416-b6be-4d11-83e2-f468a5c0bdca None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:16.439275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9e416-b6be-4d11-83e2-f468a5c0bdca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:16.439575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9e416-b6be-4d11-83e2-f468a5c0bdca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:16.441017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9e416-b6be-4d11-83e2-f468a5c0bdca None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:16.461020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 166/342] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:16.462117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3b890-5e3f-4550-b9b4-27372ea40583 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:16.462404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3b890-5e3f-4550-b9b4-27372ea40583 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:16.462678 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3b890-5e3f-4550-b9b4-27372ea40583 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:16.464009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 167/343] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:16.470553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2c498-b19c-4bb8-8d00-63229ab392ac None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:16.470834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2c498-b19c-4bb8-8d00-63229ab392ac None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:16.471113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2c498-b19c-4bb8-8d00-63229ab392ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:16.514338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-0ff2c498-b19c-4bb8-8d00-63229ab392ac None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:16.515542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0ff2c498-b19c-4bb8-8d00-63229ab392ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:16.561085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591c48c-8bc5-4083-b88e-4976e4bab569 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:16.593408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:16.645836 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f598d-0709-4f68-9af9-b88a438625f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:16.646734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f598d-0709-4f68-9af9-b88a438625f5 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:16.647190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f598d-0709-4f68-9af9-b88a438625f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:16.647634 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f598d-0709-4f68-9af9-b88a438625f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:16.669755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 168/344] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:16.670925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2c498-b19c-4bb8-8d00-63229ab392ac None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:16.692863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:16.755748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f598d-0709-4f68-9af9-b88a438625f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:16.757255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f598d-0709-4f68-9af9-b88a438625f5 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:16.883266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2c498-b19c-4bb8-8d00-63229ab392ac None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:16.954194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727162e9-c11d-4bad-b48c-6ca0ccfcecdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:16.954813 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727162e9-c11d-4bad-b48c-6ca0ccfcecdb None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:16.955095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727162e9-c11d-4bad-b48c-6ca0ccfcecdb None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:16.955366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727162e9-c11d-4bad-b48c-6ca0ccfcecdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:16.958546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 177/345] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:16.959794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add4e73-9568-4586-99a6-7436a3b80ce0 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:16.960101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add4e73-9568-4586-99a6-7436a3b80ce0 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:16.960377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add4e73-9568-4586-99a6-7436a3b80ce0 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:16.962314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 178/346] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:16.965109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add4e73-9568-4586-99a6-7436a3b80ce0 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:17.019176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:17.050683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:17.061846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727162e9-c11d-4bad-b48c-6ca0ccfcecdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:17.063306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727162e9-c11d-4bad-b48c-6ca0ccfcecdb None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:17.135508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add4e73-9568-4586-99a6-7436a3b80ce0 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:17.191607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc752cd-9929-4a0f-9ed0-70815a2f87ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:17.192284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc752cd-9929-4a0f-9ed0-70815a2f87ba None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:17.192572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc752cd-9929-4a0f-9ed0-70815a2f87ba None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:17.192847 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc752cd-9929-4a0f-9ed0-70815a2f87ba None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:17.194159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc752cd-9929-4a0f-9ed0-70815a2f87ba None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:17.197234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc752cd-9929-4a0f-9ed0-70815a2f87ba None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:17.198102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 169/347] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:17.199571 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88dda7-9812-487f-a0c4-652e3a755802 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:17.200140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88dda7-9812-487f-a0c4-652e3a755802 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:17.200345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 179/348] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:16 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:17.200881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88dda7-9812-487f-a0c4-652e3a755802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:17.201865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1447-51d3-466c-b904-c4015243603e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:17.202149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1447-51d3-466c-b904-c4015243603e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:17.202421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1447-51d3-466c-b904-c4015243603e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:17.204711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 180/349] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:17.205685 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a1447-51d3-466c-b904-c4015243603e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:17.222041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-2a88dda7-9812-487f-a0c4-652e3a755802 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:17.223825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-2a88dda7-9812-487f-a0c4-652e3a755802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:17.298883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:17.307268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 170/350] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:17.308621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88dda7-9812-487f-a0c4-652e3a755802 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:17.342565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a1447-51d3-466c-b904-c4015243603e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:17.363747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:17.391100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf3441-9aa0-4d09-8b2a-c341ff8fbc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:17.392292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3441-9aa0-4d09-8b2a-c341ff8fbc3d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:17.393083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3441-9aa0-4d09-8b2a-c341ff8fbc3d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:17.393824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3441-9aa0-4d09-8b2a-c341ff8fbc3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:17.411532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf3441-9aa0-4d09-8b2a-c341ff8fbc3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-97cf3441-9aa0-4d09-8b2a-c341ff8fbc3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:17.417432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:17.418360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:17.419231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 181/351] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:17.420176 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf3441-9aa0-4d09-8b2a-c341ff8fbc3d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:17.454872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88dda7-9812-487f-a0c4-652e3a755802 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:17.505280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91a592-059b-4115-ae00-0a6e99fe3f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:17.505947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91a592-059b-4115-ae00-0a6e99fe3f94 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:17.506238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91a592-059b-4115-ae00-0a6e99fe3f94 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:17.506660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91a592-059b-4115-ae00-0a6e99fe3f94 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:17.508806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91a592-059b-4115-ae00-0a6e99fe3f94 None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:17.511461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91a592-059b-4115-ae00-0a6e99fe3f94 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:17.516562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 171/352] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:17 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:17.517319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134baf-0664-46a3-8591-55079702e5e7 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:17.517802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134baf-0664-46a3-8591-55079702e5e7 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:17.518116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134baf-0664-46a3-8591-55079702e5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:17.539619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-2f134baf-0664-46a3-8591-55079702e5e7 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:17.540861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-2f134baf-0664-46a3-8591-55079702e5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:17.559258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf3441-9aa0-4d09-8b2a-c341ff8fbc3d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:17.608481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d13dc5-a94d-4625-bed2-1a9f27fa53db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:17.609128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d13dc5-a94d-4625-bed2-1a9f27fa53db None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:17.609475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d13dc5-a94d-4625-bed2-1a9f27fa53db None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:17.609754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d13dc5-a94d-4625-bed2-1a9f27fa53db None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:17.618723 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d13dc5-a94d-4625-bed2-1a9f27fa53db None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:17.620638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d13dc5-a94d-4625-bed2-1a9f27fa53db None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:17.622837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 182/353] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:17 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:17.624361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064d86c-0f02-4dc3-b827-d7e3663cebba None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:17.624875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064d86c-0f02-4dc3-b827-d7e3663cebba None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:17.625360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064d86c-0f02-4dc3-b827-d7e3663cebba None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:17.626856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 183/354] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:17.628008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064d86c-0f02-4dc3-b827-d7e3663cebba None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:17.632650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:17.642387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 172/355] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:17.646948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f134baf-0664-46a3-8591-55079702e5e7 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:17.683219 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:17.759600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064d86c-0f02-4dc3-b827-d7e3663cebba None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:17.774041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f134baf-0664-46a3-8591-55079702e5e7 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:17.805400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43755b3d-ac89-467c-8925-3225c6dd0952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:17.805400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43755b3d-ac89-467c-8925-3225c6dd0952 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:17.806460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43755b3d-ac89-467c-8925-3225c6dd0952 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:17.806688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43755b3d-ac89-467c-8925-3225c6dd0952 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:17.807501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43755b3d-ac89-467c-8925-3225c6dd0952 None admin] RBAC: Adding query filter params (name=member) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:17.807807 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43755b3d-ac89-467c-8925-3225c6dd0952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:17.809331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43755b3d-ac89-467c-8925-3225c6dd0952 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:17.821287 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 184/356] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:17.822097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3779c15-c21e-4e35-a66f-42050e90450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:17.822582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43755b3d-ac89-467c-8925-3225c6dd0952 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:17.822909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3779c15-c21e-4e35-a66f-42050e90450a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:17.823373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3779c15-c21e-4e35-a66f-42050e90450a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:17.823566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3779c15-c21e-4e35-a66f-42050e90450a None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:17.830532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3779c15-c21e-4e35-a66f-42050e90450a None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:17.832123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3779c15-c21e-4e35-a66f-42050e90450a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:17.834031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 173/357] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:17 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:17.834695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3779c15-c21e-4e35-a66f-42050e90450a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:17.914767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:17.950844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43755b3d-ac89-467c-8925-3225c6dd0952 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:17.952558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:17.959936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3779c15-c21e-4e35-a66f-42050e90450a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:17.989275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44ddc5-af45-4121-86a2-c88159d10b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:17.990615 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ddc5-af45-4121-86a2-c88159d10b2b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:17.990903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ddc5-af45-4121-86a2-c88159d10b2b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:17.991180 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ddc5-af45-4121-86a2-c88159d10b2b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:18.003780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44ddc5-af45-4121-86a2-c88159d10b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:18.004031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e785f7-9911-47e9-b67d-577f56ccfc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:18.004465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e785f7-9911-47e9-b67d-577f56ccfc69 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:18.004748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e785f7-9911-47e9-b67d-577f56ccfc69 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:18.005024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e785f7-9911-47e9-b67d-577f56ccfc69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:18.005888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e785f7-9911-47e9-b67d-577f56ccfc69 None admin] RBAC: Adding query filter params (group.id=3536a71c5d9f4607887f871d1729830a, role.id=0166494127b14615900b2c121908b2ba, scope.project.id=f1c8960e806347658b410f7278fe7ffd) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:18.006079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e785f7-9911-47e9-b67d-577f56ccfc69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:18.007524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e785f7-9911-47e9-b67d-577f56ccfc69 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-2f44ddc5-af45-4121-86a2-c88159d10b2b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:18.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:18.010282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:18.011226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 185/358] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:31:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:18.011984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f254d-9795-4333-af5e-7203b9743853 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:18.011984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f254d-9795-4333-af5e-7203b9743853 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:18.012389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f254d-9795-4333-af5e-7203b9743853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:18.019733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 174/359] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Aug 26 15:31:17 2022] GET /identity/v3/role_assignments?group.id=3536a71c5d9f4607887f871d1729830a&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=0166494127b14615900b2c121908b2ba => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:18.020602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e785f7-9911-47e9-b67d-577f56ccfc69 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:18.036637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e14f254d-9795-4333-af5e-7203b9743853 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:18.037889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e14f254d-9795-4333-af5e-7203b9743853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:18.135946 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 186/360] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:18.135946 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f254d-9795-4333-af5e-7203b9743853 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:18.198372 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e785f7-9911-47e9-b67d-577f56ccfc69 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:18.232481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:18.244427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05f6c8-382a-48f4-9e2c-86b05ff52625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:18.251208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05f6c8-382a-48f4-9e2c-86b05ff52625 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:18.251208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05f6c8-382a-48f4-9e2c-86b05ff52625 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:18.251208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05f6c8-382a-48f4-9e2c-86b05ff52625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:18.252213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:18.264846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f254d-9795-4333-af5e-7203b9743853 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:18.308979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0175a-7054-4217-a93d-809c0726027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:18.310048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0175a-7054-4217-a93d-809c0726027f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:18.310543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0175a-7054-4217-a93d-809c0726027f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:18.311014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0175a-7054-4217-a93d-809c0726027f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:18.312087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0175a-7054-4217-a93d-809c0726027f None admin] RBAC: Adding query filter params (group.id=3536a71c5d9f4607887f871d1729830a, role.id=e0822035b245457eade1325bbe888913, scope.project.id=f1c8960e806347658b410f7278fe7ffd) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:18.312708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0175a-7054-4217-a93d-809c0726027f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:18.314396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0175a-7054-4217-a93d-809c0726027f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:18.320936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 187/361] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Aug 26 15:31:18 2022] GET /identity/v3/role_assignments?group.id=3536a71c5d9f4607887f871d1729830a&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=e0822035b245457eade1325bbe888913 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:18.321983 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0175a-7054-4217-a93d-809c0726027f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:18.322206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05f6c8-382a-48f4-9e2c-86b05ff52625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:18.323825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05f6c8-382a-48f4-9e2c-86b05ff52625 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:18.438923 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 175/362] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:18.439991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05f6c8-382a-48f4-9e2c-86b05ff52625 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:18.440404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0175a-7054-4217-a93d-809c0726027f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:18.479538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd1f68-8a69-49cd-80d0-f7491ed89332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:18.480587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd1f68-8a69-49cd-80d0-f7491ed89332 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:18.481073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd1f68-8a69-49cd-80d0-f7491ed89332 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:18.481563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd1f68-8a69-49cd-80d0-f7491ed89332 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:18.488299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:18.494928 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd1f68-8a69-49cd-80d0-f7491ed89332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-43fd1f68-8a69-49cd-80d0-f7491ed89332 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:18.500809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:18.501806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:31:18.502642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:18.506616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 188/363] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:18.507837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd1f68-8a69-49cd-80d0-f7491ed89332 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:18.533586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:18.553347 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05f6c8-382a-48f4-9e2c-86b05ff52625 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:18.606547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f2758-12d1-43cb-8318-a66c1d8142e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:18.607454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f2758-12d1-43cb-8318-a66c1d8142e2 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:18.607937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f2758-12d1-43cb-8318-a66c1d8142e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:18.608610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f2758-12d1-43cb-8318-a66c1d8142e2 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:18.610378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f2758-12d1-43cb-8318-a66c1d8142e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:18.610848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f2758-12d1-43cb-8318-a66c1d8142e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:18.612433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f2758-12d1-43cb-8318-a66c1d8142e2 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:18.624663 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 176/364] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:18.628812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f2758-12d1-43cb-8318-a66c1d8142e2 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:18.658432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd1f68-8a69-49cd-80d0-f7491ed89332 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:18.709057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273d76b-cec4-49ae-8611-411ca0547279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:18.709706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273d76b-cec4-49ae-8611-411ca0547279 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:18.709995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273d76b-cec4-49ae-8611-411ca0547279 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:18.710278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273d76b-cec4-49ae-8611-411ca0547279 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:18.730177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273d76b-cec4-49ae-8611-411ca0547279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-7273d76b-cec4-49ae-8611-411ca0547279 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:18.736453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:18.737565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:31:18.738599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:18.742875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 189/365] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:31:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:18.743593 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273d76b-cec4-49ae-8611-411ca0547279 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:18.746930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:18.762927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f2758-12d1-43cb-8318-a66c1d8142e2 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:18.807627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55987d73-7938-4780-b2a3-66009bc700ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:18.808686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55987d73-7938-4780-b2a3-66009bc700ec None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:18.809187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55987d73-7938-4780-b2a3-66009bc700ec None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:18.809681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55987d73-7938-4780-b2a3-66009bc700ec None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:18.811080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55987d73-7938-4780-b2a3-66009bc700ec None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:18.812847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55987d73-7938-4780-b2a3-66009bc700ec None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:18.815033 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 177/366] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:18 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:18.816090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55987d73-7938-4780-b2a3-66009bc700ec None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:18.852362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:18.875958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273d76b-cec4-49ae-8611-411ca0547279 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:18.916135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fa0de-11f7-48a1-8c92-e481d10a129a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:18.917178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fa0de-11f7-48a1-8c92-e481d10a129a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:18.917670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fa0de-11f7-48a1-8c92-e481d10a129a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:18.918183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fa0de-11f7-48a1-8c92-e481d10a129a None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:18.919049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fa0de-11f7-48a1-8c92-e481d10a129a None admin] RBAC: Adding query filter params (name=reader) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:18.919562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fa0de-11f7-48a1-8c92-e481d10a129a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:18.921300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fa0de-11f7-48a1-8c92-e481d10a129a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:18.931992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 190/367] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:18.933114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fa0de-11f7-48a1-8c92-e481d10a129a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:18.943179 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55987d73-7938-4780-b2a3-66009bc700ec None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:18.990201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78764a8a-fe30-4fde-ae1c-dfd3365f9359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:18.991235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78764a8a-fe30-4fde-ae1c-dfd3365f9359 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:18.991532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78764a8a-fe30-4fde-ae1c-dfd3365f9359 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:18.991817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78764a8a-fe30-4fde-ae1c-dfd3365f9359 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:19.025320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:19.050848 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fa0de-11f7-48a1-8c92-e481d10a129a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:19.083368 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78764a8a-fe30-4fde-ae1c-dfd3365f9359 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:19.085149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78764a8a-fe30-4fde-ae1c-dfd3365f9359 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:19.095926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67d30f-440b-45be-9fd8-a668d7db4f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:19.096543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67d30f-440b-45be-9fd8-a668d7db4f3d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:19.096831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67d30f-440b-45be-9fd8-a668d7db4f3d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:19.097112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67d30f-440b-45be-9fd8-a668d7db4f3d None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:19.097786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67d30f-440b-45be-9fd8-a668d7db4f3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:19.098094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67d30f-440b-45be-9fd8-a668d7db4f3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:19.099472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67d30f-440b-45be-9fd8-a668d7db4f3d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:19.108221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 191/368] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:19.109061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67d30f-440b-45be-9fd8-a668d7db4f3d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:19.111040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:19.193956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 178/369] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:19.195743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78764a8a-fe30-4fde-ae1c-dfd3365f9359 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:19.220728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67d30f-440b-45be-9fd8-a668d7db4f3d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:19.261151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007f3b0-e5f9-4439-8bf9-76710f35c7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:19.266443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007f3b0-e5f9-4439-8bf9-76710f35c7b0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:19.266736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007f3b0-e5f9-4439-8bf9-76710f35c7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:19.267070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007f3b0-e5f9-4439-8bf9-76710f35c7b0 None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:19.268377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007f3b0-e5f9-4439-8bf9-76710f35c7b0 None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:19.269844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007f3b0-e5f9-4439-8bf9-76710f35c7b0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:19.271438 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 192/370] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:19 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:19.272234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007f3b0-e5f9-4439-8bf9-76710f35c7b0 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:19.288942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:19.344974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78764a8a-fe30-4fde-ae1c-dfd3365f9359 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:19.346595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:19.384651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ca5da-1d7c-47c7-b322-7cb8b6d19fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:19.385302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca5da-1d7c-47c7-b322-7cb8b6d19fef None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:19.385589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca5da-1d7c-47c7-b322-7cb8b6d19fef None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:19.385870 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca5da-1d7c-47c7-b322-7cb8b6d19fef None admin] PATH_INFO: `/v3/users/32c3302597ab4cd38c4550fda4d6dfad` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:19.386243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007f3b0-e5f9-4439-8bf9-76710f35c7b0 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:19.387059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ca5da-1d7c-47c7-b322-7cb8b6d19fef None admin] RBAC: Authorizing `identity:get_user(user_id=32c3302597ab4cd38c4550fda4d6dfad)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:19.388562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ca5da-1d7c-47c7-b322-7cb8b6d19fef None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:19.390246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 179/371] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:31:19 2022] GET /identity/v3/users/32c3302597ab4cd38c4550fda4d6dfad => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:19.391110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ca5da-1d7c-47c7-b322-7cb8b6d19fef None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:19.424873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108a780-99e0-4e5a-a3bd-ee7234263e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:19.425893 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108a780-99e0-4e5a-a3bd-ee7234263e3e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:19.426380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108a780-99e0-4e5a-a3bd-ee7234263e3e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:19.426858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108a780-99e0-4e5a-a3bd-ee7234263e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:19.502500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108a780-99e0-4e5a-a3bd-ee7234263e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:19.502826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ca5da-1d7c-47c7-b322-7cb8b6d19fef None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:19.504528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108a780-99e0-4e5a-a3bd-ee7234263e3e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:19.532833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:19.547214 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e8e42-94e0-4da5-8040-c3b0fb28e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:19.547905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e8e42-94e0-4da5-8040-c3b0fb28e885 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:19.548183 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e8e42-94e0-4da5-8040-c3b0fb28e885 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:19.548462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e8e42-94e0-4da5-8040-c3b0fb28e885 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:19.549895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e8e42-94e0-4da5-8040-c3b0fb28e885 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:19.551117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e8e42-94e0-4da5-8040-c3b0fb28e885 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:19.555603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 180/372] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:19 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:19.556908 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e8e42-94e0-4da5-8040-c3b0fb28e885 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:19.590498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:19.640940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 193/373] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:19.641604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108a780-99e0-4e5a-a3bd-ee7234263e3e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:19.716709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e8e42-94e0-4da5-8040-c3b0fb28e885 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:19.766654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2817ba-4209-4b94-83bd-f19c553d117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:19.767649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2817ba-4209-4b94-83bd-f19c553d117c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:19.768249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2817ba-4209-4b94-83bd-f19c553d117c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:19.768723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2817ba-4209-4b94-83bd-f19c553d117c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:19.769943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2817ba-4209-4b94-83bd-f19c553d117c None admin] RBAC: Adding query filter params (group.id=3536a71c5d9f4607887f871d1729830a, role.id=0166494127b14615900b2c121908b2ba, scope.project.id=1ee18025f6d34d0daf407c44c5f28280) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:19.770901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2817ba-4209-4b94-83bd-f19c553d117c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:19.772812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2817ba-4209-4b94-83bd-f19c553d117c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:19.773387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108a780-99e0-4e5a-a3bd-ee7234263e3e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:19.780626 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 181/374] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Aug 26 15:31:19 2022] GET /identity/v3/role_assignments?group.id=3536a71c5d9f4607887f871d1729830a&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=0166494127b14615900b2c121908b2ba => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:19.781891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2817ba-4209-4b94-83bd-f19c553d117c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:19.814796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d21eef-09ec-4ab2-86b3-f693ee424e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:19.815525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d21eef-09ec-4ab2-86b3-f693ee424e0a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:19.815894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d21eef-09ec-4ab2-86b3-f693ee424e0a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:19.816260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d21eef-09ec-4ab2-86b3-f693ee424e0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:19.817296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d21eef-09ec-4ab2-86b3-f693ee424e0a None admin] RBAC: Adding query filter params (role.id=1c9432f222d8471bb13d0479297cf5d5, user.id=32c3302597ab4cd38c4550fda4d6dfad, scope.system=all) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:19.817406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d21eef-09ec-4ab2-86b3-f693ee424e0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:19.818979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d21eef-09ec-4ab2-86b3-f693ee424e0a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:19.835829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 194/375] 10.222.0.98 () {56 vars in 1308 bytes} [Fri Aug 26 15:31:19 2022] GET /identity/v3/role_assignments?user.id=32c3302597ab4cd38c4550fda4d6dfad&scope.system=all&role.id=1c9432f222d8471bb13d0479297cf5d5 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:19.837023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d21eef-09ec-4ab2-86b3-f693ee424e0a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:19.876080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:19.879179 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:19.949776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2817ba-4209-4b94-83bd-f19c553d117c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:20.000720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d554b2-65ea-4c47-9dae-29e6d45109d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:20.002213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d554b2-65ea-4c47-9dae-29e6d45109d0 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:20.002747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d554b2-65ea-4c47-9dae-29e6d45109d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:20.003215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d554b2-65ea-4c47-9dae-29e6d45109d0 None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:20.005412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d554b2-65ea-4c47-9dae-29e6d45109d0 None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:20.007224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d554b2-65ea-4c47-9dae-29e6d45109d0 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:20.010777 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 182/376] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:19 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:20.016458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d554b2-65ea-4c47-9dae-29e6d45109d0 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:20.034373 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d21eef-09ec-4ab2-86b3-f693ee424e0a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:20.102641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f6bf6-582b-4e71-af28-38f1314337e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:20.103411 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f6bf6-582b-4e71-af28-38f1314337e2 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:20.103769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f6bf6-582b-4e71-af28-38f1314337e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:20.104129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f6bf6-582b-4e71-af28-38f1314337e2 None admin] PATH_INFO: `/v3/projects/86d26d8b55fb4a2b877ae676d9b1d379` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:20.109017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f6bf6-582b-4e71-af28-38f1314337e2 None admin] RBAC: Authorizing `identity:get_project(project_id=86d26d8b55fb4a2b877ae676d9b1d379)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:20.110719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f6bf6-582b-4e71-af28-38f1314337e2 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:20.112934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 195/377] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:19 2022] GET /identity/v3/projects/86d26d8b55fb4a2b877ae676d9b1d379 => generated 344 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:20.119031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f6bf6-582b-4e71-af28-38f1314337e2 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:20.154448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d554b2-65ea-4c47-9dae-29e6d45109d0 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:20.186583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:20.201338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda44ca0-e37c-4a34-af2e-b3e8735efe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:20.204076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda44ca0-e37c-4a34-af2e-b3e8735efe17 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:20.204076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda44ca0-e37c-4a34-af2e-b3e8735efe17 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:20.204076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda44ca0-e37c-4a34-af2e-b3e8735efe17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:20.204916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda44ca0-e37c-4a34-af2e-b3e8735efe17 None admin] RBAC: Adding query filter params (group.id=3536a71c5d9f4607887f871d1729830a, role.id=e0822035b245457eade1325bbe888913, scope.project.id=1ee18025f6d34d0daf407c44c5f28280) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:20.205175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda44ca0-e37c-4a34-af2e-b3e8735efe17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:20.207147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda44ca0-e37c-4a34-af2e-b3e8735efe17 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:20.215423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 183/378] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Aug 26 15:31:20 2022] GET /identity/v3/role_assignments?group.id=3536a71c5d9f4607887f871d1729830a&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=e0822035b245457eade1325bbe888913 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:20.252843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:20.261370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f6bf6-582b-4e71-af28-38f1314337e2 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:20.303819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aefdc3-fd4c-4ae2-bd3d-2520cdb78131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:20.305103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aefdc3-fd4c-4ae2-bd3d-2520cdb78131 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:20.305400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aefdc3-fd4c-4ae2-bd3d-2520cdb78131 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:20.305677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aefdc3-fd4c-4ae2-bd3d-2520cdb78131 None admin] PATH_INFO: `/v3/groups/26e19298f40e4d098d84d1f763da705f` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:20.313867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aefdc3-fd4c-4ae2-bd3d-2520cdb78131 None admin] RBAC: Authorizing `identity:get_group(group_id=26e19298f40e4d098d84d1f763da705f)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:20.315508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aefdc3-fd4c-4ae2-bd3d-2520cdb78131 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:20.317624 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 196/379] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:20 2022] GET /identity/v3/groups/26e19298f40e4d098d84d1f763da705f => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:20.323767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda44ca0-e37c-4a34-af2e-b3e8735efe17 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:20.451304 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda44ca0-e37c-4a34-af2e-b3e8735efe17 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:20.495247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805339b-7285-43db-8972-fcb77d6f829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:20.495906 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805339b-7285-43db-8972-fcb77d6f829b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:20.496190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805339b-7285-43db-8972-fcb77d6f829b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:20.496475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805339b-7285-43db-8972-fcb77d6f829b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:20.497219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805339b-7285-43db-8972-fcb77d6f829b None admin] RBAC: Adding query filter params (group.id=26e19298f40e4d098d84d1f763da705f, role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=86d26d8b55fb4a2b877ae676d9b1d379) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:20.497534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805339b-7285-43db-8972-fcb77d6f829b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:20.499035 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805339b-7285-43db-8972-fcb77d6f829b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:20.511996 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 184/380] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Aug 26 15:31:20 2022] GET /identity/v3/role_assignments?group.id=26e19298f40e4d098d84d1f763da705f&scope.project.id=86d26d8b55fb4a2b877ae676d9b1d379&role.id=6ba5a5add2174983be37258aad1ee404 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:20.569083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910bc68-eeef-42f3-a15a-60c7507e9fad None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:20.569425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910bc68-eeef-42f3-a15a-60c7507e9fad None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:20.569650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910bc68-eeef-42f3-a15a-60c7507e9fad None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:20.571943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 197/381] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:20.578019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8597-1888-4e2e-b0ed-173b9cfa69c4 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:20.578327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8597-1888-4e2e-b0ed-173b9cfa69c4 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:20.578577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8597-1888-4e2e-b0ed-173b9cfa69c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:20.601248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-a91b8597-1888-4e2e-b0ed-173b9cfa69c4 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:20.603144 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a91b8597-1888-4e2e-b0ed-173b9cfa69c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:20.671854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:20.700365 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 185/382] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:20.853652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06baf9-d2c7-4f6c-b05c-cdf1b0ed11a2 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:20.853866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06baf9-d2c7-4f6c-b05c-cdf1b0ed11a2 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:20.854409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06baf9-d2c7-4f6c-b05c-cdf1b0ed11a2 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:20.856260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 198/383] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:20.861311 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01e0f6-a290-4b9f-826e-14209ba27186 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:20.861593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01e0f6-a290-4b9f-826e-14209ba27186 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:20.861866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01e0f6-a290-4b9f-826e-14209ba27186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:20.892877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-1e01e0f6-a290-4b9f-826e-14209ba27186 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:20.894077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-1e01e0f6-a290-4b9f-826e-14209ba27186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:20.982414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 186/384] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:21.104416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06baf9-d2c7-4f6c-b05c-cdf1b0ed11a2 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:21.233821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06baf9-d2c7-4f6c-b05c-cdf1b0ed11a2 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:21.274554 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cc47d-de87-4d84-a02a-b4e137649c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:21.275171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cc47d-de87-4d84-a02a-b4e137649c41 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:21.275513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cc47d-de87-4d84-a02a-b4e137649c41 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:21.275798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cc47d-de87-4d84-a02a-b4e137649c41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:21.291023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445cc47d-de87-4d84-a02a-b4e137649c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-445cc47d-de87-4d84-a02a-b4e137649c41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:21.295904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:21.296808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:21.298201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 199/385] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:21.302890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01e0f6-a290-4b9f-826e-14209ba27186 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:21.324498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cc47d-de87-4d84-a02a-b4e137649c41 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:21.356299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:21.494481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cc47d-de87-4d84-a02a-b4e137649c41 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:21.499782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01e0f6-a290-4b9f-826e-14209ba27186 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:21.525893 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:21.538021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb828adf-04f2-4ed8-b64c-3323573da33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:21.538955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb828adf-04f2-4ed8-b64c-3323573da33c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:21.539988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb828adf-04f2-4ed8-b64c-3323573da33c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:21.540430 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb828adf-04f2-4ed8-b64c-3323573da33c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:21.563374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb828adf-04f2-4ed8-b64c-3323573da33c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-bb828adf-04f2-4ed8-b64c-3323573da33c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:21.569221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:21.570110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:21.571491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:21.572646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bac29-7c7c-4500-8607-cfcd78909f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:21.573212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bac29-7c7c-4500-8607-cfcd78909f7c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:21.573493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bac29-7c7c-4500-8607-cfcd78909f7c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:21.573763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bac29-7c7c-4500-8607-cfcd78909f7c None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:21.574430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bac29-7c7c-4500-8607-cfcd78909f7c None admin] RBAC: Adding query filter params (name=member) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:21.574729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bac29-7c7c-4500-8607-cfcd78909f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:21.575371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 200/386] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:31:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:21.578541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bac29-7c7c-4500-8607-cfcd78909f7c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:21.580702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb828adf-04f2-4ed8-b64c-3323573da33c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:21.590614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 187/387] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:21.596170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bac29-7c7c-4500-8607-cfcd78909f7c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:21.706381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:21.718835 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bac29-7c7c-4500-8607-cfcd78909f7c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:21.755747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb828adf-04f2-4ed8-b64c-3323573da33c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:21.764046 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c47e6-0f11-4938-9ee6-f076d85563bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:21.764688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c47e6-0f11-4938-9ee6-f076d85563bc None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:21.764967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c47e6-0f11-4938-9ee6-f076d85563bc None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:21.765241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c47e6-0f11-4938-9ee6-f076d85563bc None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:21.766373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c47e6-0f11-4938-9ee6-f076d85563bc None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:21.767797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c47e6-0f11-4938-9ee6-f076d85563bc None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:21.769686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 188/388] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:21 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:21.780904 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c47e6-0f11-4938-9ee6-f076d85563bc None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:21.808565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0d0fa-9ca5-4098-95d0-fdcfe2f2afe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:21.809180 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0d0fa-9ca5-4098-95d0-fdcfe2f2afe7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:21.809464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0d0fa-9ca5-4098-95d0-fdcfe2f2afe7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:21.809740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0d0fa-9ca5-4098-95d0-fdcfe2f2afe7 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:21.810422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f0d0fa-9ca5-4098-95d0-fdcfe2f2afe7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:21.810727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f0d0fa-9ca5-4098-95d0-fdcfe2f2afe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:21.812123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f0d0fa-9ca5-4098-95d0-fdcfe2f2afe7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:21.829345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 201/389] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:21.835280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0d0fa-9ca5-4098-95d0-fdcfe2f2afe7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:21.863888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:22.008648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c47e6-0f11-4938-9ee6-f076d85563bc None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:22.026724 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0d0fa-9ca5-4098-95d0-fdcfe2f2afe7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:22.060650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506db578-2a43-4342-9d94-abbfaff898cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:22.061284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506db578-2a43-4342-9d94-abbfaff898cf None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:22.061571 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506db578-2a43-4342-9d94-abbfaff898cf None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:22.061850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506db578-2a43-4342-9d94-abbfaff898cf None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:22.062971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506db578-2a43-4342-9d94-abbfaff898cf None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:22.064453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506db578-2a43-4342-9d94-abbfaff898cf None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:22.066126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 202/390] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:21 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:22.072234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506db578-2a43-4342-9d94-abbfaff898cf None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:22.083287 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:22.104580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d18f6d-50cd-41ba-90d7-811593727c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:22.109287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d18f6d-50cd-41ba-90d7-811593727c02 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:22.109287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d18f6d-50cd-41ba-90d7-811593727c02 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:22.109287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d18f6d-50cd-41ba-90d7-811593727c02 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:22.109287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d18f6d-50cd-41ba-90d7-811593727c02 None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:22.109287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d18f6d-50cd-41ba-90d7-811593727c02 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:22.109959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 189/391] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:21 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:22.115619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d18f6d-50cd-41ba-90d7-811593727c02 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:22.182795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506db578-2a43-4342-9d94-abbfaff898cf None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:22.228879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fa77b-06e0-4adf-ac65-4030f2c2d9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:22.229515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fa77b-06e0-4adf-ac65-4030f2c2d9ad None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:22.229794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fa77b-06e0-4adf-ac65-4030f2c2d9ad None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:22.230068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fa77b-06e0-4adf-ac65-4030f2c2d9ad None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:22.233660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fa77b-06e0-4adf-ac65-4030f2c2d9ad None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:22.235183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fa77b-06e0-4adf-ac65-4030f2c2d9ad None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:22.242732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 203/392] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:22 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:22.248145 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fa77b-06e0-4adf-ac65-4030f2c2d9ad None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:22.310469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d18f6d-50cd-41ba-90d7-811593727c02 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:22.331623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:22.366677 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81226bfc-47fb-4077-83ec-be7375120d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:22.367944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81226bfc-47fb-4077-83ec-be7375120d17 None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:22.368233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81226bfc-47fb-4077-83ec-be7375120d17 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:22.368514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81226bfc-47fb-4077-83ec-be7375120d17 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd/groups/3536a71c5d9f4607887f871d1729830a/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:22.371021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81226bfc-47fb-4077-83ec-be7375120d17 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1c8960e806347658b410f7278fe7ffd, group_id=3536a71c5d9f4607887f871d1729830a, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:22.372522 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81226bfc-47fb-4077-83ec-be7375120d17 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:22.384166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:22.406619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fa77b-06e0-4adf-ac65-4030f2c2d9ad None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:22.442291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bdd21-78bf-4ed8-b090-289e819c3426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:22.442962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bdd21-78bf-4ed8-b090-289e819c3426 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:22.443251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bdd21-78bf-4ed8-b090-289e819c3426 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:22.443584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bdd21-78bf-4ed8-b090-289e819c3426 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd/groups/3536a71c5d9f4607887f871d1729830a/roles/e0822035b245457eade1325bbe888913` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:22.444557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 190/393] 10.222.0.98 () {58 vars in 1442 bytes} [Fri Aug 26 15:31:22 2022] PUT /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd/groups/3536a71c5d9f4607887f871d1729830a/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:31:22.446903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bdd21-78bf-4ed8-b090-289e819c3426 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1c8960e806347658b410f7278fe7ffd, group_id=3536a71c5d9f4607887f871d1729830a, role_id=e0822035b245457eade1325bbe888913)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:22.447238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2d5e9-b5c1-4231-a69c-9f779acbe637 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:22.447786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2d5e9-b5c1-4231-a69c-9f779acbe637 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:22.448360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bdd21-78bf-4ed8-b090-289e819c3426 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:22.448478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2d5e9-b5c1-4231-a69c-9f779acbe637 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:22.450180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 191/394] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:22.456447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011204-225a-47b1-8ad3-0e993853bdc9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:22.457168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011204-225a-47b1-8ad3-0e993853bdc9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:22.457686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011204-225a-47b1-8ad3-0e993853bdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:22.480503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-9a011204-225a-47b1-8ad3-0e993853bdc9 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:22.482268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9a011204-225a-47b1-8ad3-0e993853bdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:22.560152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 204/395] 10.222.0.98 () {58 vars in 1442 bytes} [Fri Aug 26 15:31:22 2022] PUT /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd/groups/3536a71c5d9f4607887f871d1729830a/roles/e0822035b245457eade1325bbe888913 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:31:22.571112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 192/396] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:22.873414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c566d-8efb-488b-ac3e-bc509de383a3 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:22.873751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c566d-8efb-488b-ac3e-bc509de383a3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:22.873926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c566d-8efb-488b-ac3e-bc509de383a3 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:22.875369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 205/397] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:22.880855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1bb79-bfb9-4906-be1c-b17d7e4ee12b None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:22.881455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1bb79-bfb9-4906-be1c-b17d7e4ee12b None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:22.881961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1bb79-bfb9-4906-be1c-b17d7e4ee12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:22.892854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218d8ec-0ec2-45c7-bbbd-1c8ff1d49372 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:22.893439 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218d8ec-0ec2-45c7-bbbd-1c8ff1d49372 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:22.893920 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218d8ec-0ec2-45c7-bbbd-1c8ff1d49372 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:22.895481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 206/398] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:22.900129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218d8ec-0ec2-45c7-bbbd-1c8ff1d49372 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:22.906169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-2dd1bb79-bfb9-4906-be1c-b17d7e4ee12b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:22.907357 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-2dd1bb79-bfb9-4906-be1c-b17d7e4ee12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:22.983894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 193/399] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:22.983894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf2007-e5c3-41e9-9669-3421d4d7ead3 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:22.983894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf2007-e5c3-41e9-9669-3421d4d7ead3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:22.983894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf2007-e5c3-41e9-9669-3421d4d7ead3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:23.017068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-0edf2007-e5c3-41e9-9669-3421d4d7ead3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:23.018387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0edf2007-e5c3-41e9-9669-3421d4d7ead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:23.047446 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:23.059901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218d8ec-0ec2-45c7-bbbd-1c8ff1d49372 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:23.096954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680fa4a-3fd0-4d62-aabf-a1ce3f8ec185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:23.097590 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680fa4a-3fd0-4d62-aabf-a1ce3f8ec185 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:23.097875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680fa4a-3fd0-4d62-aabf-a1ce3f8ec185 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:23.098152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680fa4a-3fd0-4d62-aabf-a1ce3f8ec185 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:23.100922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:23.109761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680fa4a-3fd0-4d62-aabf-a1ce3f8ec185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-0680fa4a-3fd0-4d62-aabf-a1ce3f8ec185 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:23.114461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:23.115448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:23.116358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 207/400] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:23.123094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680fa4a-3fd0-4d62-aabf-a1ce3f8ec185 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:23.167102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 194/401] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:23.231551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680fa4a-3fd0-4d62-aabf-a1ce3f8ec185 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:23.266577 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a1f3d-5ba7-41e2-8175-4d780da7440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:23.267177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a1f3d-5ba7-41e2-8175-4d780da7440f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:23.267462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a1f3d-5ba7-41e2-8175-4d780da7440f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:23.267827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a1f3d-5ba7-41e2-8175-4d780da7440f None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:23.268525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a1f3d-5ba7-41e2-8175-4d780da7440f None admin] RBAC: Adding query filter params (name=member) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:23.268831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a1f3d-5ba7-41e2-8175-4d780da7440f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:23.270213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a1f3d-5ba7-41e2-8175-4d780da7440f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:23.279071 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 208/402] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:23.284787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf2007-e5c3-41e9-9669-3421d4d7ead3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:23.287620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a1f3d-5ba7-41e2-8175-4d780da7440f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:23.360363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:23.404930 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a1f3d-5ba7-41e2-8175-4d780da7440f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:23.413573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf2007-e5c3-41e9-9669-3421d4d7ead3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:23.454040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008c296-dc64-4385-9125-ed381d538953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:23.454683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008c296-dc64-4385-9125-ed381d538953 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:23.454969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008c296-dc64-4385-9125-ed381d538953 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:23.455249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008c296-dc64-4385-9125-ed381d538953 None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:23.456903 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008c296-dc64-4385-9125-ed381d538953 None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:23.458349 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008c296-dc64-4385-9125-ed381d538953 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:23.465652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 195/403] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:23 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:23.466994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008c296-dc64-4385-9125-ed381d538953 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:23.476711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b79fdf-5d19-4329-b4ea-91519639fec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:23.477742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b79fdf-5d19-4329-b4ea-91519639fec4 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:23.478288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b79fdf-5d19-4329-b4ea-91519639fec4 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:23.478759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b79fdf-5d19-4329-b4ea-91519639fec4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:23.495436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b79fdf-5d19-4329-b4ea-91519639fec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:23.497547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-c7b79fdf-5d19-4329-b4ea-91519639fec4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:23.502153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:23.503037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:23.504012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:23.506572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 209/404] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:31:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:23.512980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b79fdf-5d19-4329-b4ea-91519639fec4 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:23.597788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008c296-dc64-4385-9125-ed381d538953 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:23.639785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7cad2-dbef-4752-a003-242576eccf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:23.640870 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7cad2-dbef-4752-a003-242576eccf93 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:23.641861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7cad2-dbef-4752-a003-242576eccf93 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:23.642336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7cad2-dbef-4752-a003-242576eccf93 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:23.644171 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7cad2-dbef-4752-a003-242576eccf93 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:23.646596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7cad2-dbef-4752-a003-242576eccf93 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:23.650228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 196/405] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:23 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:23.651245 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7cad2-dbef-4752-a003-242576eccf93 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:23.660498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b79fdf-5d19-4329-b4ea-91519639fec4 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:23.684360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:23.708261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474c972-7a52-4342-a6af-af45c40e4ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:23.708261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c972-7a52-4342-a6af-af45c40e4ce7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:23.708261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c972-7a52-4342-a6af-af45c40e4ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:23.708261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c972-7a52-4342-a6af-af45c40e4ce7 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:23.708834 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474c972-7a52-4342-a6af-af45c40e4ce7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:23.709351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474c972-7a52-4342-a6af-af45c40e4ce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:23.710916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474c972-7a52-4342-a6af-af45c40e4ce7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:23.725192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 210/406] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:23.726386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474c972-7a52-4342-a6af-af45c40e4ce7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:23.778924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:23.786502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7cad2-dbef-4752-a003-242576eccf93 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:23.833312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de088d23-5077-429a-9630-25071f7aa612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:23.834277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de088d23-5077-429a-9630-25071f7aa612 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:23.834571 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de088d23-5077-429a-9630-25071f7aa612 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:23.834853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de088d23-5077-429a-9630-25071f7aa612 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:23.841283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474c972-7a52-4342-a6af-af45c40e4ce7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:23.848364 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de088d23-5077-429a-9630-25071f7aa612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-de088d23-5077-429a-9630-25071f7aa612 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:23.854532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:23.855514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:23.856392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 197/407] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:31:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:23.857092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de088d23-5077-429a-9630-25071f7aa612 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:23.895683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbc939-9ec6-45df-994b-0f9f35a3545f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:23.896533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbc939-9ec6-45df-994b-0f9f35a3545f None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:23.896927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbc939-9ec6-45df-994b-0f9f35a3545f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:23.897222 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbc939-9ec6-45df-994b-0f9f35a3545f None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280/groups/3536a71c5d9f4607887f871d1729830a/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:23.899531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbc939-9ec6-45df-994b-0f9f35a3545f None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee18025f6d34d0daf407c44c5f28280, group_id=3536a71c5d9f4607887f871d1729830a, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:23.902014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbc939-9ec6-45df-994b-0f9f35a3545f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:23.973016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de088d23-5077-429a-9630-25071f7aa612 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:24.009078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2859a-3211-4733-85cc-3ef1cdd07531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:24.009726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2859a-3211-4733-85cc-3ef1cdd07531 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:24.009989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2859a-3211-4733-85cc-3ef1cdd07531 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:24.010263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2859a-3211-4733-85cc-3ef1cdd07531 None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:24.011385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2859a-3211-4733-85cc-3ef1cdd07531 None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:24.012886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2859a-3211-4733-85cc-3ef1cdd07531 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:24.019161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 211/409] 10.222.0.98 () {58 vars in 1442 bytes} [Fri Aug 26 15:31:23 2022] PUT /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280/groups/3536a71c5d9f4607887f871d1729830a/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:31:24.019351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 198/409] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:23 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:24.023405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2859a-3211-4733-85cc-3ef1cdd07531 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:24.024694 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbc939-9ec6-45df-994b-0f9f35a3545f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:24.077721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:24.127445 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:24.232837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2859a-3211-4733-85cc-3ef1cdd07531 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:24.244423 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbc939-9ec6-45df-994b-0f9f35a3545f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:24.271516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b876d93-c68a-42a4-a01b-1cb724bb1986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:24.272716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b876d93-c68a-42a4-a01b-1cb724bb1986 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:24.273205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b876d93-c68a-42a4-a01b-1cb724bb1986 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:24.273712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b876d93-c68a-42a4-a01b-1cb724bb1986 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:24.274697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b876d93-c68a-42a4-a01b-1cb724bb1986 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:24.275308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b876d93-c68a-42a4-a01b-1cb724bb1986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:24.277312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b876d93-c68a-42a4-a01b-1cb724bb1986 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:24.279739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e257-75c2-4ab1-89b4-d0063f2154e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:24.280402 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e257-75c2-4ab1-89b4-d0063f2154e5 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:24.280695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e257-75c2-4ab1-89b4-d0063f2154e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:24.280970 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e257-75c2-4ab1-89b4-d0063f2154e5 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:24.282914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e257-75c2-4ab1-89b4-d0063f2154e5 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:24.284403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e257-75c2-4ab1-89b4-d0063f2154e5 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:24.286034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 212/410] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:24 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:24.289744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 199/411] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:24.292892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e257-75c2-4ab1-89b4-d0063f2154e5 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:24.295042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b876d93-c68a-42a4-a01b-1cb724bb1986 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:24.395298 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:24.400945 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e257-75c2-4ab1-89b4-d0063f2154e5 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:24.412179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:24.415562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b876d93-c68a-42a4-a01b-1cb724bb1986 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:24.439850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c5d21-7231-48f9-ab7d-5b550d075c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:24.440823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c5d21-7231-48f9-ab7d-5b550d075c57 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:24.440935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c5d21-7231-48f9-ab7d-5b550d075c57 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:24.441189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c5d21-7231-48f9-ab7d-5b550d075c57 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280/groups/3536a71c5d9f4607887f871d1729830a/roles/e0822035b245457eade1325bbe888913` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:24.443558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c5d21-7231-48f9-ab7d-5b550d075c57 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee18025f6d34d0daf407c44c5f28280, group_id=3536a71c5d9f4607887f871d1729830a, role_id=e0822035b245457eade1325bbe888913)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:24.445069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c5d21-7231-48f9-ab7d-5b550d075c57 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:24.460358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343823d-0c2b-40e3-89cd-a1fb21c64df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:24.461368 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343823d-0c2b-40e3-89cd-a1fb21c64df7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:24.463489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343823d-0c2b-40e3-89cd-a1fb21c64df7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:24.463489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343823d-0c2b-40e3-89cd-a1fb21c64df7 None admin] PATH_INFO: `/v3/groups/26e19298f40e4d098d84d1f763da705f` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:24.463489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343823d-0c2b-40e3-89cd-a1fb21c64df7 None admin] RBAC: Authorizing `identity:get_group(group_id=26e19298f40e4d098d84d1f763da705f)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:24.465244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343823d-0c2b-40e3-89cd-a1fb21c64df7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:24.467353 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 200/412] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:24 2022] GET /identity/v3/groups/26e19298f40e4d098d84d1f763da705f => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:24.473139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343823d-0c2b-40e3-89cd-a1fb21c64df7 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:24.560592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 213/413] 10.222.0.98 () {58 vars in 1442 bytes} [Fri Aug 26 15:31:24 2022] PUT /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280/groups/3536a71c5d9f4607887f871d1729830a/roles/e0822035b245457eade1325bbe888913 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:31:24.594925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343823d-0c2b-40e3-89cd-a1fb21c64df7 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:24.636535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66a795-2681-40c1-9106-c47a8aa13ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:24.637241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66a795-2681-40c1-9106-c47a8aa13ef0 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:24.637491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66a795-2681-40c1-9106-c47a8aa13ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:24.637723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66a795-2681-40c1-9106-c47a8aa13ef0 None admin] PATH_INFO: `/v3/projects/86d26d8b55fb4a2b877ae676d9b1d379` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:24.638896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66a795-2681-40c1-9106-c47a8aa13ef0 None admin] RBAC: Authorizing `identity:get_project(project_id=86d26d8b55fb4a2b877ae676d9b1d379)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:24.640411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66a795-2681-40c1-9106-c47a8aa13ef0 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:24.642495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 201/414] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:24 2022] GET /identity/v3/projects/86d26d8b55fb4a2b877ae676d9b1d379 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:24.648411 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c5d21-7231-48f9-ab7d-5b550d075c57 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:24.763014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c5d21-7231-48f9-ab7d-5b550d075c57 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:24.800723 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c7a40-250b-48da-9964-ea3692b42485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:24.801393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c7a40-250b-48da-9964-ea3692b42485 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:24.801762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c7a40-250b-48da-9964-ea3692b42485 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:24.802044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c7a40-250b-48da-9964-ea3692b42485 None admin] PATH_INFO: `/v3/projects/86d26d8b55fb4a2b877ae676d9b1d379/groups/26e19298f40e4d098d84d1f763da705f/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:24.804351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c7a40-250b-48da-9964-ea3692b42485 None admin] RBAC: Authorizing `identity:create_grant(project_id=86d26d8b55fb4a2b877ae676d9b1d379, group_id=26e19298f40e4d098d84d1f763da705f, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:24.805860 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c7a40-250b-48da-9964-ea3692b42485 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:24.850259 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c6554-64a8-41cd-b238-6992774e33da None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:24.850458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c6554-64a8-41cd-b238-6992774e33da None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:24.850736 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c6554-64a8-41cd-b238-6992774e33da None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:24.853129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 202/415] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:24.857797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a2731-dad3-4b38-95e9-62761b23ff4d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:24.858085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a2731-dad3-4b38-95e9-62761b23ff4d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:24.858363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a2731-dad3-4b38-95e9-62761b23ff4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:24.876680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 214/416] 10.222.0.98 () {58 vars in 1442 bytes} [Fri Aug 26 15:31:24 2022] PUT /identity/v3/projects/86d26d8b55fb4a2b877ae676d9b1d379/groups/26e19298f40e4d098d84d1f763da705f/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:31:24.888943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b65a2731-dad3-4b38-95e9-62761b23ff4d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:24.890190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b65a2731-dad3-4b38-95e9-62761b23ff4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:24.911755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:24.979763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55980f12-76d3-436e-bd83-a642c9c8519a None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:24.980484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55980f12-76d3-436e-bd83-a642c9c8519a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:24.980967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55980f12-76d3-436e-bd83-a642c9c8519a None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:24.981271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 203/417] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:24.983101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 215/418] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:24.989041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e37e52-5454-468a-93e0-e2dce573d9d2 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:24.989692 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e37e52-5454-468a-93e0-e2dce573d9d2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:24.990174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e37e52-5454-468a-93e0-e2dce573d9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:25.011194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-46e37e52-5454-468a-93e0-e2dce573d9d2 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:25.012871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-46e37e52-5454-468a-93e0-e2dce573d9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:25.099348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 204/419] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:25.372176 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55980f12-76d3-436e-bd83-a642c9c8519a None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:25.409582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:25.434539 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e37e52-5454-468a-93e0-e2dce573d9d2 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:25.514181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55980f12-76d3-436e-bd83-a642c9c8519a None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:25.581356 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8842d7-5398-425f-a7a6-3140ff204df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:25.582017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8842d7-5398-425f-a7a6-3140ff204df9 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:25.582305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8842d7-5398-425f-a7a6-3140ff204df9 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:25.582589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8842d7-5398-425f-a7a6-3140ff204df9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:25.612599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8842d7-5398-425f-a7a6-3140ff204df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:25.616232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-3a8842d7-5398-425f-a7a6-3140ff204df9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:25.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:25.619515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:25.620380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 216/420] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:25.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8842d7-5398-425f-a7a6-3140ff204df9 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:25.665841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e37e52-5454-468a-93e0-e2dce573d9d2 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:25.732877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed8665-110b-416c-91c7-d0e40f7e714b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:25.733531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed8665-110b-416c-91c7-d0e40f7e714b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:25.733811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed8665-110b-416c-91c7-d0e40f7e714b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:25.734087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed8665-110b-416c-91c7-d0e40f7e714b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:25.750416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed8665-110b-416c-91c7-d0e40f7e714b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-15ed8665-110b-416c-91c7-d0e40f7e714b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:25.755949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:25.756839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:25.757681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 205/421] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:31:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:25.765283 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed8665-110b-416c-91c7-d0e40f7e714b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:25.797809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:25.821513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8842d7-5398-425f-a7a6-3140ff204df9 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:25.871091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f2227-e990-49ac-a9ce-d5cf1b2dea9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:25.871706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f2227-e990-49ac-a9ce-d5cf1b2dea9f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:25.872031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f2227-e990-49ac-a9ce-d5cf1b2dea9f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:25.872313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f2227-e990-49ac-a9ce-d5cf1b2dea9f None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:25.872993 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f2227-e990-49ac-a9ce-d5cf1b2dea9f None admin] RBAC: Adding query filter params (name=member) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:25.873307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f2227-e990-49ac-a9ce-d5cf1b2dea9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:25.874716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f2227-e990-49ac-a9ce-d5cf1b2dea9f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:25.884447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 217/422] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:25.890488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f2227-e990-49ac-a9ce-d5cf1b2dea9f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:25.942565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed8665-110b-416c-91c7-d0e40f7e714b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:26.009073 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f44d1-2f92-417a-bb17-3122969003a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:26.009691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f44d1-2f92-417a-bb17-3122969003a6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:26.010019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f44d1-2f92-417a-bb17-3122969003a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:26.010300 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f44d1-2f92-417a-bb17-3122969003a6 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:26.010976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f44d1-2f92-417a-bb17-3122969003a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:26.011283 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f44d1-2f92-417a-bb17-3122969003a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:26.013696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f44d1-2f92-417a-bb17-3122969003a6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:26.023624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 206/423] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:26.029318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f44d1-2f92-417a-bb17-3122969003a6 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:26.038461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:26.086441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f2227-e990-49ac-a9ce-d5cf1b2dea9f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:26.153457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af20fa-0ebe-4eb4-87c3-5fb3d3e31a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:26.153996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af20fa-0ebe-4eb4-87c3-5fb3d3e31a2c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:26.154288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af20fa-0ebe-4eb4-87c3-5fb3d3e31a2c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:26.154570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af20fa-0ebe-4eb4-87c3-5fb3d3e31a2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:26.189691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f44d1-2f92-417a-bb17-3122969003a6 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:26.197058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:26.252203 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e8b82-6ea6-411e-b6d4-45d2d666f7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:26.253208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8b82-6ea6-411e-b6d4-45d2d666f7d9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:26.253843 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8b82-6ea6-411e-b6d4-45d2d666f7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:26.254328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8b82-6ea6-411e-b6d4-45d2d666f7d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:26.310468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af20fa-0ebe-4eb4-87c3-5fb3d3e31a2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:26.312343 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af20fa-0ebe-4eb4-87c3-5fb3d3e31a2c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:26.370232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e8b82-6ea6-411e-b6d4-45d2d666f7d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:26.371738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e8b82-6ea6-411e-b6d4-45d2d666f7d9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:26.380215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:26.445903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 218/424] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:26.454290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af20fa-0ebe-4eb4-87c3-5fb3d3e31a2c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:26.506935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:26.522915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 207/425] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 15:31:26.528909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada1ba5-b58f-45de-ba05-16eb7d72a245 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:26.528909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada1ba5-b58f-45de-ba05-16eb7d72a245 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:26.528909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada1ba5-b58f-45de-ba05-16eb7d72a245 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:26.530531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 208/426] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:26.531610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada1ba5-b58f-45de-ba05-16eb7d72a245 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:26.568182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af20fa-0ebe-4eb4-87c3-5fb3d3e31a2c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:26.609940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a95fa-7199-4d5e-b9be-ad47aa7e752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:26.610825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a95fa-7199-4d5e-b9be-ad47aa7e752b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:26.611328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a95fa-7199-4d5e-b9be-ad47aa7e752b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:26.612279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a95fa-7199-4d5e-b9be-ad47aa7e752b None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:26.614065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a95fa-7199-4d5e-b9be-ad47aa7e752b None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:26.616454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a95fa-7199-4d5e-b9be-ad47aa7e752b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:26.619890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 219/427] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:26 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:26.621803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b153ac2-a73a-4156-99e6-be46205fbfe6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:26.622295 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b153ac2-a73a-4156-99e6-be46205fbfe6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:26.623198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b153ac2-a73a-4156-99e6-be46205fbfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:26.638770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada1ba5-b58f-45de-ba05-16eb7d72a245 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:26.641986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:26.647419 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9b153ac2-a73a-4156-99e6-be46205fbfe6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:26.649200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9b153ac2-a73a-4156-99e6-be46205fbfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:26.679258 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5fcfd-9d19-4430-9212-04f6be0e2a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:26.680666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5fcfd-9d19-4430-9212-04f6be0e2a69 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:26.681049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5fcfd-9d19-4430-9212-04f6be0e2a69 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:26.681425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5fcfd-9d19-4430-9212-04f6be0e2a69 None admin] PATH_INFO: `/v3/projects/f1c8960e806347658b410f7278fe7ffd` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:26.682707 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5fcfd-9d19-4430-9212-04f6be0e2a69 None admin] RBAC: Authorizing `identity:get_project(project_id=f1c8960e806347658b410f7278fe7ffd)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:26.684370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5fcfd-9d19-4430-9212-04f6be0e2a69 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:26.686114 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 209/428] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:26 2022] GET /identity/v3/projects/f1c8960e806347658b410f7278fe7ffd => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:26.687995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5fcfd-9d19-4430-9212-04f6be0e2a69 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:26.751981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 220/429] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:26.751981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b153ac2-a73a-4156-99e6-be46205fbfe6 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:26.806225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:26.843288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5fcfd-9d19-4430-9212-04f6be0e2a69 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:26.869779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b153ac2-a73a-4156-99e6-be46205fbfe6 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:26.881609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f586d-0155-4ea1-96db-445dd78c98c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:26.882586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f586d-0155-4ea1-96db-445dd78c98c9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:26.883221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f586d-0155-4ea1-96db-445dd78c98c9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:26.884279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f586d-0155-4ea1-96db-445dd78c98c9 None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:26.885632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f586d-0155-4ea1-96db-445dd78c98c9 None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:26.887603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f586d-0155-4ea1-96db-445dd78c98c9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:26.889826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 210/430] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:26 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:26.891265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa0b03-951e-452d-998d-0af75a1b393d None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:26.891778 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa0b03-951e-452d-998d-0af75a1b393d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:26.892275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa0b03-951e-452d-998d-0af75a1b393d None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:26.893873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 211/431] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:26.903525 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcacae4-77be-413a-9a13-622587e9a44e None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:26.903525 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcacae4-77be-413a-9a13-622587e9a44e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:26.903525 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcacae4-77be-413a-9a13-622587e9a44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:26.908203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b8ecd-348e-4eed-acb4-5d6e1fdc0806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:26.909660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b8ecd-348e-4eed-acb4-5d6e1fdc0806 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:26.909938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b8ecd-348e-4eed-acb4-5d6e1fdc0806 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:26.910214 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b8ecd-348e-4eed-acb4-5d6e1fdc0806 None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:26.911352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b8ecd-348e-4eed-acb4-5d6e1fdc0806 None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:26.912879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b8ecd-348e-4eed-acb4-5d6e1fdc0806 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:26.914546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 221/432] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:26 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:26.916016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b8ecd-348e-4eed-acb4-5d6e1fdc0806 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:26.926363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-cdcacae4-77be-413a-9a13-622587e9a44e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:26.927894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cdcacae4-77be-413a-9a13-622587e9a44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:26.966829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:27.008224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 212/433] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:27.009883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcacae4-77be-413a-9a13-622587e9a44e None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:27.040361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b8ecd-348e-4eed-acb4-5d6e1fdc0806 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:27.059134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:27.079296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43a727-8e57-4522-a961-38557ad961d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:27.079921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43a727-8e57-4522-a961-38557ad961d9 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:27.080209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43a727-8e57-4522-a961-38557ad961d9 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:27.080483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43a727-8e57-4522-a961-38557ad961d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:27.081204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43a727-8e57-4522-a961-38557ad961d9 None admin] RBAC: Adding query filter params (group.id=3536a71c5d9f4607887f871d1729830a, role.id=0166494127b14615900b2c121908b2ba, scope.project.id=f1c8960e806347658b410f7278fe7ffd) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:27.081511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43a727-8e57-4522-a961-38557ad961d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:27.082935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43a727-8e57-4522-a961-38557ad961d9 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:27.089119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 222/434] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Aug 26 15:31:26 2022] GET /identity/v3/role_assignments?group.id=3536a71c5d9f4607887f871d1729830a&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=0166494127b14615900b2c121908b2ba => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:27.089699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43a727-8e57-4522-a961-38557ad961d9 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:27.149154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcacae4-77be-413a-9a13-622587e9a44e None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:27.188964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1342691-4941-4ea9-bab9-81fbc7c80354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:27.189709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1342691-4941-4ea9-bab9-81fbc7c80354 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:27.190069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1342691-4941-4ea9-bab9-81fbc7c80354 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:27.190428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1342691-4941-4ea9-bab9-81fbc7c80354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:27.191256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1342691-4941-4ea9-bab9-81fbc7c80354 None admin] RBAC: Adding query filter params (group.id=3536a71c5d9f4607887f871d1729830a, role.id=e0822035b245457eade1325bbe888913, scope.project.id=f1c8960e806347658b410f7278fe7ffd) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:27.191727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1342691-4941-4ea9-bab9-81fbc7c80354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:27.193458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1342691-4941-4ea9-bab9-81fbc7c80354 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:27.202325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43a727-8e57-4522-a961-38557ad961d9 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:27.213454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 213/435] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Aug 26 15:31:27 2022] GET /identity/v3/role_assignments?group.id=3536a71c5d9f4607887f871d1729830a&scope.project.id=f1c8960e806347658b410f7278fe7ffd&role.id=e0822035b245457eade1325bbe888913 => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:27.259491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd8238-6e3e-458a-b1a4-333b0fd9a65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:27.260086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd8238-6e3e-458a-b1a4-333b0fd9a65b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:27.260371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd8238-6e3e-458a-b1a4-333b0fd9a65b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:27.260646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd8238-6e3e-458a-b1a4-333b0fd9a65b None admin] PATH_INFO: `/v3/roles/member` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:27.282292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd8238-6e3e-458a-b1a4-333b0fd9a65b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-07bd8238-6e3e-458a-b1a4-333b0fd9a65b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:27.287648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:27.288618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:31:27.290425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:27.291478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e0119-2516-4678-b4b3-383834447a3f None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:27.291478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e0119-2516-4678-b4b3-383834447a3f None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:27.291478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e0119-2516-4678-b4b3-383834447a3f None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:27.292225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 223/436] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:27.292373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 214/437] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:27.296479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e0119-2516-4678-b4b3-383834447a3f None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:27.296650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685d89-ed46-471c-a626-9dff7a70cc87 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:27.296853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685d89-ed46-471c-a626-9dff7a70cc87 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:27.297054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685d89-ed46-471c-a626-9dff7a70cc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:27.317331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e3685d89-ed46-471c-a626-9dff7a70cc87 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:27.318577 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e3685d89-ed46-471c-a626-9dff7a70cc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:27.347773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:27.359178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:27.397242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 224/438] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:27.418375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3685d89-ed46-471c-a626-9dff7a70cc87 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:27.452662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e0119-2516-4678-b4b3-383834447a3f None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:27.501764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8731d3-fa2b-4e25-833a-a624c5cb6a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:27.502654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8731d3-fa2b-4e25-833a-a624c5cb6a77 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:27.503336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8731d3-fa2b-4e25-833a-a624c5cb6a77 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:27.503763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8731d3-fa2b-4e25-833a-a624c5cb6a77 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:27.505178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8731d3-fa2b-4e25-833a-a624c5cb6a77 None admin] RBAC: Adding query filter params (name=member) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:27.505503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8731d3-fa2b-4e25-833a-a624c5cb6a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:27.507201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8731d3-fa2b-4e25-833a-a624c5cb6a77 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:27.518725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 215/439] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:31:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:27.525173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8731d3-fa2b-4e25-833a-a624c5cb6a77 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:27.556676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3685d89-ed46-471c-a626-9dff7a70cc87 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:27.613679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1d724-173b-4443-943b-462e2432a0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:27.613679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1d724-173b-4443-943b-462e2432a0b1 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:27.613679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1d724-173b-4443-943b-462e2432a0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:27.613679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1d724-173b-4443-943b-462e2432a0b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:27.636111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1d724-173b-4443-943b-462e2432a0b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-3ee1d724-173b-4443-943b-462e2432a0b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:27.642921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:27.643841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:27.644705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:27.647632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 225/440] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:31:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:27.648703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1d724-173b-4443-943b-462e2432a0b1 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:27.657954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:27.666345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8731d3-fa2b-4e25-833a-a624c5cb6a77 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:27.689641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:27.709612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe430f-13ac-40b2-81f5-0bd09f8016db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:27.710316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe430f-13ac-40b2-81f5-0bd09f8016db None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:27.710468 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe430f-13ac-40b2-81f5-0bd09f8016db None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:27.710745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe430f-13ac-40b2-81f5-0bd09f8016db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:27.778514 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1d724-173b-4443-943b-462e2432a0b1 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:27.798339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe430f-13ac-40b2-81f5-0bd09f8016db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:27.800240 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe430f-13ac-40b2-81f5-0bd09f8016db None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:27.825871 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b47ea-3a5b-40b9-9b6a-cc92e964d46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:27.826483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b47ea-3a5b-40b9-9b6a-cc92e964d46d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:27.826762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b47ea-3a5b-40b9-9b6a-cc92e964d46d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:27.827034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b47ea-3a5b-40b9-9b6a-cc92e964d46d None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:27.827735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b47ea-3a5b-40b9-9b6a-cc92e964d46d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:27.830294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b47ea-3a5b-40b9-9b6a-cc92e964d46d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:27.830294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b47ea-3a5b-40b9-9b6a-cc92e964d46d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:27.838610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 226/441] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:27.839541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b47ea-3a5b-40b9-9b6a-cc92e964d46d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:27.916427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:27.922643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 216/442] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:27.923559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe430f-13ac-40b2-81f5-0bd09f8016db None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:27.944212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:27.953107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b47ea-3a5b-40b9-9b6a-cc92e964d46d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:27.986888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b685d7-cc01-427b-a1f1-b282f37ee65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:27.987537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b685d7-cc01-427b-a1f1-b282f37ee65a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:27.987822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b685d7-cc01-427b-a1f1-b282f37ee65a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:27.988131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b685d7-cc01-427b-a1f1-b282f37ee65a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:27.998864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b685d7-cc01-427b-a1f1-b282f37ee65a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-d2b685d7-cc01-427b-a1f1-b282f37ee65a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:28.004072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:28.005097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:28.006082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 227/443] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:31:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:28.007149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b685d7-cc01-427b-a1f1-b282f37ee65a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:28.031150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe430f-13ac-40b2-81f5-0bd09f8016db None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:28.070467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f659b-23c2-40c1-a0c6-e95810eaf07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:28.071117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f659b-23c2-40c1-a0c6-e95810eaf07e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:28.071401 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f659b-23c2-40c1-a0c6-e95810eaf07e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:28.071683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f659b-23c2-40c1-a0c6-e95810eaf07e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:28.107303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b685d7-cc01-427b-a1f1-b282f37ee65a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:28.134504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:28.140156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715b174-3cf7-4e28-aae0-6d9a91902dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:28.140925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715b174-3cf7-4e28-aae0-6d9a91902dd4 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:28.141184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715b174-3cf7-4e28-aae0-6d9a91902dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:28.141425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f659b-23c2-40c1-a0c6-e95810eaf07e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:28.141526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715b174-3cf7-4e28-aae0-6d9a91902dd4 None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:28.142557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3715b174-3cf7-4e28-aae0-6d9a91902dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:28.142892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f659b-23c2-40c1-a0c6-e95810eaf07e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:28.144011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3715b174-3cf7-4e28-aae0-6d9a91902dd4 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:28.146030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 228/444] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:28 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:28.146345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715b174-3cf7-4e28-aae0-6d9a91902dd4 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:28.170152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:28.244840 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 217/445] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:28.245769 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f659b-23c2-40c1-a0c6-e95810eaf07e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:28.246785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715b174-3cf7-4e28-aae0-6d9a91902dd4 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:28.280205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc738f22-01d0-4a31-824e-66536f90201f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:28.280803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc738f22-01d0-4a31-824e-66536f90201f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:28.281086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc738f22-01d0-4a31-824e-66536f90201f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:28.281415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc738f22-01d0-4a31-824e-66536f90201f None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:28.282027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc738f22-01d0-4a31-824e-66536f90201f None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:28.282332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc738f22-01d0-4a31-824e-66536f90201f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:28.283703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc738f22-01d0-4a31-824e-66536f90201f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:28.292503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 229/446] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:28.293113 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc738f22-01d0-4a31-824e-66536f90201f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:28.344700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:28.350257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f659b-23c2-40c1-a0c6-e95810eaf07e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:28.376056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:28.389275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2b35d-d3bc-40ff-bd45-6d0b6b29dd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:28.389914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2b35d-d3bc-40ff-bd45-6d0b6b29dd6d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:28.390196 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2b35d-d3bc-40ff-bd45-6d0b6b29dd6d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:28.390472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2b35d-d3bc-40ff-bd45-6d0b6b29dd6d None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:28.391566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2b35d-d3bc-40ff-bd45-6d0b6b29dd6d None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:28.393029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2b35d-d3bc-40ff-bd45-6d0b6b29dd6d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:28.394593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 218/447] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:28 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:28.395356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2b35d-d3bc-40ff-bd45-6d0b6b29dd6d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:28.395587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc738f22-01d0-4a31-824e-66536f90201f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:28.438019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31dad-e977-4408-b59d-770c0870afae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:28.438726 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31dad-e977-4408-b59d-770c0870afae None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:28.439094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31dad-e977-4408-b59d-770c0870afae None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:28.439431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31dad-e977-4408-b59d-770c0870afae None admin] PATH_INFO: `/v3/projects/1ee18025f6d34d0daf407c44c5f28280` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:28.440597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac31dad-e977-4408-b59d-770c0870afae None admin] RBAC: Authorizing `identity:get_project(project_id=1ee18025f6d34d0daf407c44c5f28280)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:28.442041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac31dad-e977-4408-b59d-770c0870afae None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:28.443666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 230/448] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:28 2022] GET /identity/v3/projects/1ee18025f6d34d0daf407c44c5f28280 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:28.444465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31dad-e977-4408-b59d-770c0870afae None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:28.504471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2b35d-d3bc-40ff-bd45-6d0b6b29dd6d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:28.537788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c1347-dd61-4460-bf9e-606831372ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:28.538334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c1347-dd61-4460-bf9e-606831372ba0 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:28.538619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c1347-dd61-4460-bf9e-606831372ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:28.538894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c1347-dd61-4460-bf9e-606831372ba0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:28.544927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31dad-e977-4408-b59d-770c0870afae None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:28.569818 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:28.580546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b54afb-f325-4420-b501-12a5d19bd2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:28.581184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b54afb-f325-4420-b501-12a5d19bd2fa None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:28.582104 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b54afb-f325-4420-b501-12a5d19bd2fa None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:28.582104 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b54afb-f325-4420-b501-12a5d19bd2fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:28.582472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b54afb-f325-4420-b501-12a5d19bd2fa None admin] RBAC: Adding query filter params (group.id=3536a71c5d9f4607887f871d1729830a, role.id=0166494127b14615900b2c121908b2ba, scope.project.id=1ee18025f6d34d0daf407c44c5f28280) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:28.582786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b54afb-f325-4420-b501-12a5d19bd2fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:28.584261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b54afb-f325-4420-b501-12a5d19bd2fa None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:28.590348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 231/449] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Aug 26 15:31:28 2022] GET /identity/v3/role_assignments?group.id=3536a71c5d9f4607887f871d1729830a&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=0166494127b14615900b2c121908b2ba => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:28.591132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b54afb-f325-4420-b501-12a5d19bd2fa None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:28.609647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c1347-dd61-4460-bf9e-606831372ba0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:28.611292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c1347-dd61-4460-bf9e-606831372ba0 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:28.621058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:28.720485 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b54afb-f325-4420-b501-12a5d19bd2fa None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:28.726836 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 219/450] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:28.732810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c1347-dd61-4460-bf9e-606831372ba0 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:28.761040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945424d-be1d-4193-b703-2cd8293a7098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:28.762033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945424d-be1d-4193-b703-2cd8293a7098 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:28.762475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945424d-be1d-4193-b703-2cd8293a7098 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:28.762890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945424d-be1d-4193-b703-2cd8293a7098 None admin] PATH_INFO: `/v3/groups/3536a71c5d9f4607887f871d1729830a` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:28.764344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945424d-be1d-4193-b703-2cd8293a7098 None admin] RBAC: Authorizing `identity:get_group(group_id=3536a71c5d9f4607887f871d1729830a)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:28.766085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945424d-be1d-4193-b703-2cd8293a7098 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:28.767848 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 232/451] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:28 2022] GET /identity/v3/groups/3536a71c5d9f4607887f871d1729830a => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:28.774716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945424d-be1d-4193-b703-2cd8293a7098 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:28.816652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:28.861511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c1347-dd61-4460-bf9e-606831372ba0 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:28.871190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:28.897321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945424d-be1d-4193-b703-2cd8293a7098 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:28.901487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22f66e-7f4e-49b5-be58-dd4941efd9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:28.902325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22f66e-7f4e-49b5-be58-dd4941efd9c9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:28.902749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22f66e-7f4e-49b5-be58-dd4941efd9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:28.903166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22f66e-7f4e-49b5-be58-dd4941efd9c9 None admin] PATH_INFO: `/v3/projects/86d26d8b55fb4a2b877ae676d9b1d379` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:28.904472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22f66e-7f4e-49b5-be58-dd4941efd9c9 None admin] RBAC: Authorizing `identity:get_project(project_id=86d26d8b55fb4a2b877ae676d9b1d379)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:28.906023 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22f66e-7f4e-49b5-be58-dd4941efd9c9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:28.908012 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 220/452] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:31:28 2022] GET /identity/v3/projects/86d26d8b55fb4a2b877ae676d9b1d379 => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:28.914407 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22f66e-7f4e-49b5-be58-dd4941efd9c9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:28.940452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c715e4c-7855-4574-b66c-1070a098400e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:28.941065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c715e4c-7855-4574-b66c-1070a098400e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:28.941356 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c715e4c-7855-4574-b66c-1070a098400e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:28.941633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c715e4c-7855-4574-b66c-1070a098400e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:28.942769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c715e4c-7855-4574-b66c-1070a098400e None admin] RBAC: Adding query filter params (group.id=3536a71c5d9f4607887f871d1729830a, role.id=e0822035b245457eade1325bbe888913, scope.project.id=1ee18025f6d34d0daf407c44c5f28280) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:28.942769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c715e4c-7855-4574-b66c-1070a098400e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:28.944155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c715e4c-7855-4574-b66c-1070a098400e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:28.950690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 233/453] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Aug 26 15:31:28 2022] GET /identity/v3/role_assignments?group.id=3536a71c5d9f4607887f871d1729830a&scope.project.id=1ee18025f6d34d0daf407c44c5f28280&role.id=e0822035b245457eade1325bbe888913 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:29.036222 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22f66e-7f4e-49b5-be58-dd4941efd9c9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:29.067621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:29.072646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220a2d1-ca52-4d7f-89d3-bbcb96f1f4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:29.073300 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220a2d1-ca52-4d7f-89d3-bbcb96f1f4ca None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:29.073584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220a2d1-ca52-4d7f-89d3-bbcb96f1f4ca None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:29.073863 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220a2d1-ca52-4d7f-89d3-bbcb96f1f4ca None admin] PATH_INFO: `/v3/groups/26e19298f40e4d098d84d1f763da705f` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:29.075024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220a2d1-ca52-4d7f-89d3-bbcb96f1f4ca None admin] RBAC: Authorizing `identity:get_group(group_id=26e19298f40e4d098d84d1f763da705f)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:29.076527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220a2d1-ca52-4d7f-89d3-bbcb96f1f4ca None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:29.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 221/454] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Aug 26 15:31:28 2022] GET /identity/v3/groups/26e19298f40e4d098d84d1f763da705f => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:29.084389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c715e4c-7855-4574-b66c-1070a098400e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:29.193469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c715e4c-7855-4574-b66c-1070a098400e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:29.226674 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619b169-f67e-4f11-b598-7b8bdef9bbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:29.227835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619b169-f67e-4f11-b598-7b8bdef9bbf1 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:29.228155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619b169-f67e-4f11-b598-7b8bdef9bbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:29.228431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619b169-f67e-4f11-b598-7b8bdef9bbf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:29.229162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619b169-f67e-4f11-b598-7b8bdef9bbf1 None admin] RBAC: Adding query filter params (group.id=26e19298f40e4d098d84d1f763da705f, role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=86d26d8b55fb4a2b877ae676d9b1d379) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:29.229474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619b169-f67e-4f11-b598-7b8bdef9bbf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:29.230914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619b169-f67e-4f11-b598-7b8bdef9bbf1 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:29.236853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 234/455] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Aug 26 15:31:29 2022] GET /identity/v3/role_assignments?group.id=26e19298f40e4d098d84d1f763da705f&scope.project.id=86d26d8b55fb4a2b877ae676d9b1d379&role.id=6ba5a5add2174983be37258aad1ee404 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:31.759760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e646-0af0-4117-9ec5-9adfd079eaa6 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:31.760296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e646-0af0-4117-9ec5-9adfd079eaa6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:31.760369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e646-0af0-4117-9ec5-9adfd079eaa6 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:31.761978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 222/456] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:31.777774 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2a785-7165-4a34-a010-3ecc494455d1 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:31.778275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2a785-7165-4a34-a010-3ecc494455d1 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:31.778360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2a785-7165-4a34-a010-3ecc494455d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:31.802344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e0f2a785-7165-4a34-a010-3ecc494455d1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:31.803567 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e0f2a785-7165-4a34-a010-3ecc494455d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:31.807465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:31.813243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9127e35-f0a5-4d13-914b-1074f8b80f26 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:31.813538 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9127e35-f0a5-4d13-914b-1074f8b80f26 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:31.813826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9127e35-f0a5-4d13-914b-1074f8b80f26 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:31.815390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 223/457] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:31.820695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d33db5-ea20-4d19-8b48-4f7bba83688b None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:31.821005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d33db5-ea20-4d19-8b48-4f7bba83688b None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:31.821328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d33db5-ea20-4d19-8b48-4f7bba83688b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:31.848191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-27d33db5-ea20-4d19-8b48-4f7bba83688b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:31.853420 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-27d33db5-ea20-4d19-8b48-4f7bba83688b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:31.906689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 235/458] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:31.964161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 224/459] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:31.972981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610c63d-40e6-4d7c-b2ae-997615fd15dc None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:31.973267 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610c63d-40e6-4d7c-b2ae-997615fd15dc None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:31.973544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610c63d-40e6-4d7c-b2ae-997615fd15dc None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:31.975022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 236/460] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:31.980486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3826f29-9fcf-4f70-8993-c9f8d87de449 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:31.980771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3826f29-9fcf-4f70-8993-c9f8d87de449 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:31.981055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3826f29-9fcf-4f70-8993-c9f8d87de449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:32.004974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c3826f29-9fcf-4f70-8993-c9f8d87de449 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:32.006159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c3826f29-9fcf-4f70-8993-c9f8d87de449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:32.076229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 225/461] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:32.218907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610c63d-40e6-4d7c-b2ae-997615fd15dc None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:32.319843 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3826f29-9fcf-4f70-8993-c9f8d87de449 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:32.351024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610c63d-40e6-4d7c-b2ae-997615fd15dc None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:32.387128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68649997-6293-41b8-ae19-5371040caefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:32.387750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649997-6293-41b8-ae19-5371040caefe None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:32.388060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649997-6293-41b8-ae19-5371040caefe None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:32.388335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649997-6293-41b8-ae19-5371040caefe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:32.396683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:32.400120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68649997-6293-41b8-ae19-5371040caefe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:32.402174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68649997-6293-41b8-ae19-5371040caefe None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:32.402694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-68649997-6293-41b8-ae19-5371040caefe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:32.426055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:32.427045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:32.432985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:32.432985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:32.432985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:32.432985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 237/462] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:32.432985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68649997-6293-41b8-ae19-5371040caefe None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:32.458381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3826f29-9fcf-4f70-8993-c9f8d87de449 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:32.502115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a1f30-b222-4638-8855-cb9f9de02283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:32.503073 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a1f30-b222-4638-8855-cb9f9de02283 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:32.503513 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a1f30-b222-4638-8855-cb9f9de02283 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:32.504057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a1f30-b222-4638-8855-cb9f9de02283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:32.521160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a1f30-b222-4638-8855-cb9f9de02283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:32.523521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a1f30-b222-4638-8855-cb9f9de02283 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-e24a1f30-b222-4638-8855-cb9f9de02283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:32.542547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:32.543516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:32.545064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:32.545064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:32.545064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:32.545064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 226/463] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:32.545064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a1f30-b222-4638-8855-cb9f9de02283 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:32.554720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68649997-6293-41b8-ae19-5371040caefe None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:32.594526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39686a-2e20-4604-9b0e-2959a997b360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:32.595149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39686a-2e20-4604-9b0e-2959a997b360 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:32.595430 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39686a-2e20-4604-9b0e-2959a997b360 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:32.595702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39686a-2e20-4604-9b0e-2959a997b360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:32.613243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39686a-2e20-4604-9b0e-2959a997b360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:32.615010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39686a-2e20-4604-9b0e-2959a997b360 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-ed39686a-2e20-4604-9b0e-2959a997b360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:32.633301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:32.634781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:32.635630 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:32.635630 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:32.635630 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:32.636515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 238/464] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:32.637892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f8d12-7ca4-40bf-87f3-b11ab83adbd5 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:32.638181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f8d12-7ca4-40bf-87f3-b11ab83adbd5 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:32.638455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f8d12-7ca4-40bf-87f3-b11ab83adbd5 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:32.639900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 239/465] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:32.644528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f8d12-7ca4-40bf-87f3-b11ab83adbd5 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:32.658103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a1f30-b222-4638-8855-cb9f9de02283 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:32.671993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:32.695345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:32.697675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab526b2-4967-4a2d-b16b-058227ec2b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:32.698592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab526b2-4967-4a2d-b16b-058227ec2b08 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:32.698955 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab526b2-4967-4a2d-b16b-058227ec2b08 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:32.699306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab526b2-4967-4a2d-b16b-058227ec2b08 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:32.700079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab526b2-4967-4a2d-b16b-058227ec2b08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:32.700454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab526b2-4967-4a2d-b16b-058227ec2b08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:32.702099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab526b2-4967-4a2d-b16b-058227ec2b08 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:32.718654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 227/466] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:32.719669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab526b2-4967-4a2d-b16b-058227ec2b08 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:32.761152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f8d12-7ca4-40bf-87f3-b11ab83adbd5 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:32.796007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6eccf-96db-4b32-8dd0-a7b99f254fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:32.796664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6eccf-96db-4b32-8dd0-a7b99f254fcf None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:32.796664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6eccf-96db-4b32-8dd0-a7b99f254fcf None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:32.796897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6eccf-96db-4b32-8dd0-a7b99f254fcf None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:32.797537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6eccf-96db-4b32-8dd0-a7b99f254fcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:32.797841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6eccf-96db-4b32-8dd0-a7b99f254fcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:32.799257 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6eccf-96db-4b32-8dd0-a7b99f254fcf None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:32.816658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 240/467] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:32.817830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94645579-09bc-4d13-93fc-f1b02288afb3 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:32.818119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94645579-09bc-4d13-93fc-f1b02288afb3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:32.818396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94645579-09bc-4d13-93fc-f1b02288afb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:32.832399 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab526b2-4967-4a2d-b16b-058227ec2b08 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:32.836766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-94645579-09bc-4d13-93fc-f1b02288afb3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:32.838465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-94645579-09bc-4d13-93fc-f1b02288afb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:32.869461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bf881-e619-46ae-8521-717a88448f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:32.870064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bf881-e619-46ae-8521-717a88448f5c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:32.870353 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bf881-e619-46ae-8521-717a88448f5c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:32.870692 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bf881-e619-46ae-8521-717a88448f5c None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:32.871346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227bf881-e619-46ae-8521-717a88448f5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:32.871656 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227bf881-e619-46ae-8521-717a88448f5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:32.873084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227bf881-e619-46ae-8521-717a88448f5c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:32.885813 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 228/468] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:32.886719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bf881-e619-46ae-8521-717a88448f5c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:32.923911 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 241/469] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:32.923911 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94645579-09bc-4d13-93fc-f1b02288afb3 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:32.936343 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:32.960851 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:33.032064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bf881-e619-46ae-8521-717a88448f5c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:33.051396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94645579-09bc-4d13-93fc-f1b02288afb3 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:33.070473 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2947b0e-651d-4f17-9c86-ba14e175e676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:33.071998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2947b0e-651d-4f17-9c86-ba14e175e676 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:33.072690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2947b0e-651d-4f17-9c86-ba14e175e676 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:33.072690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2947b0e-651d-4f17-9c86-ba14e175e676 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:33.073389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2947b0e-651d-4f17-9c86-ba14e175e676 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:33.075178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2947b0e-651d-4f17-9c86-ba14e175e676 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:33.096592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd8438-9529-4724-808f-61bd8e9b7a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:33.097311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd8438-9529-4724-808f-61bd8e9b7a04 None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:33.097657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd8438-9529-4724-808f-61bd8e9b7a04 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:33.098004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd8438-9529-4724-808f-61bd8e9b7a04 None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:33.098811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd8438-9529-4724-808f-61bd8e9b7a04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:33.100364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd8438-9529-4724-808f-61bd8e9b7a04 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:33.186698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 229/470] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:31:32 2022] POST /identity/v3/users => generated 246 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:31:33.187603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2947b0e-651d-4f17-9c86-ba14e175e676 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:33.228730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 242/471] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:31:32 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:31:33.231281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c9790-34d6-43d4-bf54-02fb4d870e62 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:33.231752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c9790-34d6-43d4-bf54-02fb4d870e62 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:33.232429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c9790-34d6-43d4-bf54-02fb4d870e62 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:33.238938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 243/472] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:33.240227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c9790-34d6-43d4-bf54-02fb4d870e62 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:33.322954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:33.346501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2947b0e-651d-4f17-9c86-ba14e175e676 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:33.375165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:33.391091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c9790-34d6-43d4-bf54-02fb4d870e62 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:33.409948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce392c95-07ab-471e-a9a7-4a1c434679b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:33.410728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce392c95-07ab-471e-a9a7-4a1c434679b9 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:33.410959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce392c95-07ab-471e-a9a7-4a1c434679b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:33.411238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce392c95-07ab-471e-a9a7-4a1c434679b9 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:33.411910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce392c95-07ab-471e-a9a7-4a1c434679b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:33.414683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce392c95-07ab-471e-a9a7-4a1c434679b9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:33.436864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7e2cd-8835-440a-930f-f90fb2da5fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:33.437498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7e2cd-8835-440a-930f-f90fb2da5fa0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:33.437782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7e2cd-8835-440a-930f-f90fb2da5fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:33.438059 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7e2cd-8835-440a-930f-f90fb2da5fa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:33.448519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7e2cd-8835-440a-930f-f90fb2da5fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:33.449984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7e2cd-8835-440a-930f-f90fb2da5fa0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-90d7e2cd-8835-440a-930f-f90fb2da5fa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:33.465516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:33.466458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:33.467552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:33.467552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:33.467552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:33.467552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 244/473] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:33.467780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c8ab-f73b-4dc9-98f9-663cbb7f3968 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:33.468177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c8ab-f73b-4dc9-98f9-663cbb7f3968 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:33.468386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c8ab-f73b-4dc9-98f9-663cbb7f3968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:33.489201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-b4b8c8ab-f73b-4dc9-98f9-663cbb7f3968 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:33.490627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-b4b8c8ab-f73b-4dc9-98f9-663cbb7f3968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:33.572058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 245/474] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:33.572927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8c8ab-f73b-4dc9-98f9-663cbb7f3968 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:33.630944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 230/475] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:31:33 2022] POST /identity/v3/users => generated 248 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:31:33.651093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:33.685867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8c8ab-f73b-4dc9-98f9-663cbb7f3968 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:33.737918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae46393-6715-4f80-8686-66564b30bc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:33.739171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae46393-6715-4f80-8686-66564b30bc5d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:33.739673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae46393-6715-4f80-8686-66564b30bc5d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:33.740352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae46393-6715-4f80-8686-66564b30bc5d None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:33.741649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae46393-6715-4f80-8686-66564b30bc5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:33.742435 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae46393-6715-4f80-8686-66564b30bc5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:33.745592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae46393-6715-4f80-8686-66564b30bc5d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:33.763271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 246/476] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:33.774211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce392c95-07ab-471e-a9a7-4a1c434679b9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:33.902383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae46393-6715-4f80-8686-66564b30bc5d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:33.917003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce392c95-07ab-471e-a9a7-4a1c434679b9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:33.954952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f79f39-0d2a-43c8-b062-d4ec594528cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:33.955573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f79f39-0d2a-43c8-b062-d4ec594528cc None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:33.955859 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f79f39-0d2a-43c8-b062-d4ec594528cc None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:33.956173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f79f39-0d2a-43c8-b062-d4ec594528cc None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:33.956820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f79f39-0d2a-43c8-b062-d4ec594528cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:33.958266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f79f39-0d2a-43c8-b062-d4ec594528cc None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:34.022184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae46393-6715-4f80-8686-66564b30bc5d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:34.046024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:34.050938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 231/477] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:31:33 2022] POST /identity/v3/users => generated 249 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:31:34.067732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:34.074924 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788a08c-7c2f-479a-bc69-56da6555fa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:34.075563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788a08c-7c2f-479a-bc69-56da6555fa3c None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:34.075849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788a08c-7c2f-479a-bc69-56da6555fa3c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:34.076160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788a08c-7c2f-479a-bc69-56da6555fa3c None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:34.076975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788a08c-7c2f-479a-bc69-56da6555fa3c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:34.078535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788a08c-7c2f-479a-bc69-56da6555fa3c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:34.116108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.sql.core [None req-5788a08c-7c2f-479a-bc69-56da6555fa3c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 15:31:34.116108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 15:31:34.116108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [parameters: {'id': '0ac2353fc61a4e9c8fb55f9c9f702692', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 15:31:34.116108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82214) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-5788a08c-7c2f-479a-bc69-56da6555fa3c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application result.read() Aug 26 15:31:34.141357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return ref.to_dict() Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:31:34.142376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application yield resource Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise self.value Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application session.commit() Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 15:31:34.143383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self.session.flush() Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application compat.raise_( Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 15:31:34.144330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:31:34.146339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application result.read() Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:31:34.147212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [parameters: {'id': '0ac2353fc61a4e9c8fb55f9c9f702692', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:34.151382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 15:31:34.153557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:34.155620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 247/478] 10.222.0.98 () {60 vars in 1131 bytes} [Fri Aug 26 15:31:33 2022] POST /identity/v3/roles => generated 144 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:34.155768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f79f39-0d2a-43c8-b062-d4ec594528cc None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:34.284356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f79f39-0d2a-43c8-b062-d4ec594528cc None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:34.327948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d518d33-6128-4673-9a11-2725b6ca5800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:34.328973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d518d33-6128-4673-9a11-2725b6ca5800 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:34.329465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d518d33-6128-4673-9a11-2725b6ca5800 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:34.329941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d518d33-6128-4673-9a11-2725b6ca5800 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:34.344556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d518d33-6128-4673-9a11-2725b6ca5800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-5d518d33-6128-4673-9a11-2725b6ca5800 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:34.350162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:34.351083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:31:34.351983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:34.355120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 232/479] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:31:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:34.360532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788a08c-7c2f-479a-bc69-56da6555fa3c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:34.483505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788a08c-7c2f-479a-bc69-56da6555fa3c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:34.521953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac650c-cb4e-4e74-b7c3-3ae9ba869e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:34.522950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac650c-cb4e-4e74-b7c3-3ae9ba869e4d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:34.523446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac650c-cb4e-4e74-b7c3-3ae9ba869e4d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:34.523954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac650c-cb4e-4e74-b7c3-3ae9ba869e4d None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:34.525382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac650c-cb4e-4e74-b7c3-3ae9ba869e4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:34.525894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac650c-cb4e-4e74-b7c3-3ae9ba869e4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:34.527501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac650c-cb4e-4e74-b7c3-3ae9ba869e4d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:34.538910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 248/480] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:35.556440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713fba4-ef0d-4536-a33a-874fec26bce2 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:35.556959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713fba4-ef0d-4536-a33a-874fec26bce2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:35.556959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713fba4-ef0d-4536-a33a-874fec26bce2 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:35.558566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 233/481] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:35.564139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377ad4f-7e24-4688-a4fd-1efcb9d0cea0 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:35.564553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377ad4f-7e24-4688-a4fd-1efcb9d0cea0 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:35.564748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377ad4f-7e24-4688-a4fd-1efcb9d0cea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:35.583615 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-a377ad4f-7e24-4688-a4fd-1efcb9d0cea0 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:35.584842 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a377ad4f-7e24-4688-a4fd-1efcb9d0cea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:35.665819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:35.682382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 249/482] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:35.723851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d528ca9-2792-4379-9537-7890600b302e None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:35.724162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d528ca9-2792-4379-9537-7890600b302e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:35.724447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d528ca9-2792-4379-9537-7890600b302e None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:35.725995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 234/483] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:35.732440 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24e23-0af9-443f-bdbd-e7be8fd02f06 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:35.732767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24e23-0af9-443f-bdbd-e7be8fd02f06 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:35.733051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24e23-0af9-443f-bdbd-e7be8fd02f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:35.761484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-72c24e23-0af9-443f-bdbd-e7be8fd02f06 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:35.762759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-72c24e23-0af9-443f-bdbd-e7be8fd02f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:35.854641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 250/484] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:36.034671 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d737724-bac0-42c3-a6fe-2bed2f4d08b3 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:36.035162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d737724-bac0-42c3-a6fe-2bed2f4d08b3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:36.035545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d737724-bac0-42c3-a6fe-2bed2f4d08b3 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:36.037280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 235/485] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:36.043305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425af00f-9024-4124-883a-bbe386031768 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:36.043622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425af00f-9024-4124-883a-bbe386031768 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:36.043885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425af00f-9024-4124-883a-bbe386031768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:36.066300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-425af00f-9024-4124-883a-bbe386031768 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:36.067484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-425af00f-9024-4124-883a-bbe386031768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:36.103321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d737724-bac0-42c3-a6fe-2bed2f4d08b3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:36.185477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 251/486] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:36.216656 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:36.233829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425af00f-9024-4124-883a-bbe386031768 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:36.281429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d737724-bac0-42c3-a6fe-2bed2f4d08b3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:36.328234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:36.349953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39b97e-b884-4902-b9c5-5956c6bf67ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:36.350598 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39b97e-b884-4902-b9c5-5956c6bf67ae None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:36.350881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39b97e-b884-4902-b9c5-5956c6bf67ae None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:36.352472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39b97e-b884-4902-b9c5-5956c6bf67ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:36.376961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39b97e-b884-4902-b9c5-5956c6bf67ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-5e39b97e-b884-4902-b9c5-5956c6bf67ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:36.384345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:36.385248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:36.386069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 236/487] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:36.392097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39b97e-b884-4902-b9c5-5956c6bf67ae None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:36.420779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425af00f-9024-4124-883a-bbe386031768 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:36.491294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c73b98-ce0a-4186-93c7-9ad91382bc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:36.492077 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c73b98-ce0a-4186-93c7-9ad91382bc11 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:36.492366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c73b98-ce0a-4186-93c7-9ad91382bc11 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:36.492646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c73b98-ce0a-4186-93c7-9ad91382bc11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:36.512498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c73b98-ce0a-4186-93c7-9ad91382bc11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-06c73b98-ce0a-4186-93c7-9ad91382bc11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:36.517417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:36.518441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:36.519250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 252/488] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:36.519984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c73b98-ce0a-4186-93c7-9ad91382bc11 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:36.559915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39b97e-b884-4902-b9c5-5956c6bf67ae None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:36.590816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:36.610403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3a327-431f-4ce2-8b7e-d66e6b05c9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:36.612527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a327-431f-4ce2-8b7e-d66e6b05c9d3 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:36.612527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a327-431f-4ce2-8b7e-d66e6b05c9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:36.612527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a327-431f-4ce2-8b7e-d66e6b05c9d3 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:36.613210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3a327-431f-4ce2-8b7e-d66e6b05c9d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:36.613581 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3a327-431f-4ce2-8b7e-d66e6b05c9d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:36.615360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3a327-431f-4ce2-8b7e-d66e6b05c9d3 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:36.628475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 237/489] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:36.628475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3a327-431f-4ce2-8b7e-d66e6b05c9d3 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:36.644300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c73b98-ce0a-4186-93c7-9ad91382bc11 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:36.655148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:36.683808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c998b-586c-4146-b5dd-07700b88fb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:36.684865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c998b-586c-4146-b5dd-07700b88fb5c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:36.685333 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c998b-586c-4146-b5dd-07700b88fb5c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:36.685691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c998b-586c-4146-b5dd-07700b88fb5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:36.699722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c998b-586c-4146-b5dd-07700b88fb5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-f79c998b-586c-4146-b5dd-07700b88fb5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:36.704487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:36.705416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 253/490] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:36.708787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81737a5-416c-4752-9bd5-aca1f4611400 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:36.713297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81737a5-416c-4752-9bd5-aca1f4611400 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:36.713297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81737a5-416c-4752-9bd5-aca1f4611400 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:36.713297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 254/491] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:36.713297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81737a5-416c-4752-9bd5-aca1f4611400 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:36.738067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3a327-431f-4ce2-8b7e-d66e6b05c9d3 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:36.774076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315790a0-a640-4259-b94e-3b8450c1ed53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:36.775045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315790a0-a640-4259-b94e-3b8450c1ed53 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:36.775565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315790a0-a640-4259-b94e-3b8450c1ed53 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:36.776060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315790a0-a640-4259-b94e-3b8450c1ed53 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:36.776912 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315790a0-a640-4259-b94e-3b8450c1ed53 None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:36.777386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315790a0-a640-4259-b94e-3b8450c1ed53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:36.778948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315790a0-a640-4259-b94e-3b8450c1ed53 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:36.787920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 238/492] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:36.789053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315790a0-a640-4259-b94e-3b8450c1ed53 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:36.830248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81737a5-416c-4752-9bd5-aca1f4611400 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:36.836515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:36.867040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09485ef-a0bf-47b7-8552-0d11db193311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:36.867666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09485ef-a0bf-47b7-8552-0d11db193311 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:36.867971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09485ef-a0bf-47b7-8552-0d11db193311 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:36.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09485ef-a0bf-47b7-8552-0d11db193311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:36.879197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09485ef-a0bf-47b7-8552-0d11db193311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:36.880815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09485ef-a0bf-47b7-8552-0d11db193311 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-c09485ef-a0bf-47b7-8552-0d11db193311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:36.896394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:36.897342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:36.898220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:36.898220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:36.898220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:36.898220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 255/493] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:36.901169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b441e-db19-4f0e-b649-7633183187df None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:36.901458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b441e-db19-4f0e-b649-7633183187df None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:36.901767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b441e-db19-4f0e-b649-7633183187df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:36.924446 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315790a0-a640-4259-b94e-3b8450c1ed53 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:36.927155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ff1b441e-db19-4f0e-b649-7633183187df None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:36.928395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ff1b441e-db19-4f0e-b649-7633183187df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:36.943674 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:36.962294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475fe55-30da-4269-b387-821ced1d31b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:36.962908 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475fe55-30da-4269-b387-821ced1d31b9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:36.963195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475fe55-30da-4269-b387-821ced1d31b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:36.963474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475fe55-30da-4269-b387-821ced1d31b9 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:36.964198 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475fe55-30da-4269-b387-821ced1d31b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:36.967379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475fe55-30da-4269-b387-821ced1d31b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:36.967379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475fe55-30da-4269-b387-821ced1d31b9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:36.976228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 239/494] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:36.978325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475fe55-30da-4269-b387-821ced1d31b9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:36.999120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 256/495] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:37.000005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b441e-db19-4f0e-b649-7633183187df None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:37.098213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475fe55-30da-4269-b387-821ced1d31b9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:37.106686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:37.139035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b441e-db19-4f0e-b649-7633183187df None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:37.139355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfd90e-57ea-4ede-a859-c3468bf01876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:37.140138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfd90e-57ea-4ede-a859-c3468bf01876 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:37.140515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfd90e-57ea-4ede-a859-c3468bf01876 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:37.140886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfd90e-57ea-4ede-a859-c3468bf01876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:37.153616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfd90e-57ea-4ede-a859-c3468bf01876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:37.155266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfd90e-57ea-4ede-a859-c3468bf01876 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-0edfd90e-57ea-4ede-a859-c3468bf01876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:37.173318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:37.174531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:37.175569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:37.175569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:37.175569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:37.176409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 240/496] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:37.181529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfd90e-57ea-4ede-a859-c3468bf01876 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:37.185558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce240b8-e77e-4dd2-bfd7-a5d5cebacffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:37.186276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce240b8-e77e-4dd2-bfd7-a5d5cebacffa None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:37.186654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce240b8-e77e-4dd2-bfd7-a5d5cebacffa None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:37.187034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce240b8-e77e-4dd2-bfd7-a5d5cebacffa None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:37.187774 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce240b8-e77e-4dd2-bfd7-a5d5cebacffa None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:37.188208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce240b8-e77e-4dd2-bfd7-a5d5cebacffa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:37.189672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce240b8-e77e-4dd2-bfd7-a5d5cebacffa None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:37.202105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 257/497] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:37.207511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf0b7-aedd-4916-80b5-9f53c018ba59 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:37.207801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf0b7-aedd-4916-80b5-9f53c018ba59 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:37.208210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf0b7-aedd-4916-80b5-9f53c018ba59 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:37.209397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 258/498] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:37.210021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabf0b7-aedd-4916-80b5-9f53c018ba59 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:37.223140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:37.298934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfd90e-57ea-4ede-a859-c3468bf01876 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:37.316229 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabf0b7-aedd-4916-80b5-9f53c018ba59 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:37.332429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51863f-0f4d-41ed-8276-38c9a3637b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:37.333062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51863f-0f4d-41ed-8276-38c9a3637b8c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:37.333348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51863f-0f4d-41ed-8276-38c9a3637b8c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:37.333683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51863f-0f4d-41ed-8276-38c9a3637b8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:37.344123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51863f-0f4d-41ed-8276-38c9a3637b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:37.345562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51863f-0f4d-41ed-8276-38c9a3637b8c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:37.350612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081411b-e043-45c8-9f30-c5a110c95f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:37.351152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081411b-e043-45c8-9f30-c5a110c95f53 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:37.351432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081411b-e043-45c8-9f30-c5a110c95f53 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:37.351710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081411b-e043-45c8-9f30-c5a110c95f53 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:37.352370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081411b-e043-45c8-9f30-c5a110c95f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:37.353235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081411b-e043-45c8-9f30-c5a110c95f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:37.354581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081411b-e043-45c8-9f30-c5a110c95f53 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-0d51863f-0f4d-41ed-8276-38c9a3637b8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:37.359936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:37.361738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:37.363723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:37.363723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:37.363723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:37.363723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 241/499] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:37.363723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51863f-0f4d-41ed-8276-38c9a3637b8c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:37.370254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 259/500] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:37.371563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a26b7-961d-48ff-a87d-ec2718b46842 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:37.371922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a26b7-961d-48ff-a87d-ec2718b46842 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:37.372261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a26b7-961d-48ff-a87d-ec2718b46842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:37.375747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:37.390695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-858a26b7-961d-48ff-a87d-ec2718b46842 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:37.392687 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-858a26b7-961d-48ff-a87d-ec2718b46842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:37.467946 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:37.472668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 260/501] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:37.473384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a26b7-961d-48ff-a87d-ec2718b46842 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:37.495074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51863f-0f4d-41ed-8276-38c9a3637b8c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:37.536526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc44434-6482-4731-9c69-df8ebd502d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:37.537383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44434-6482-4731-9c69-df8ebd502d74 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:37.537781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44434-6482-4731-9c69-df8ebd502d74 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:37.538148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44434-6482-4731-9c69-df8ebd502d74 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:37.548527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc44434-6482-4731-9c69-df8ebd502d74 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:37.550363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc44434-6482-4731-9c69-df8ebd502d74 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-0dc44434-6482-4731-9c69-df8ebd502d74 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:37.569106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:37.572181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 242/502] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:37.575820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc44434-6482-4731-9c69-df8ebd502d74 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:37.591810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a26b7-961d-48ff-a87d-ec2718b46842 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:37.632939 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1a9e2-7cd4-4e65-8acc-8ab84fc62452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:37.633786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1a9e2-7cd4-4e65-8acc-8ab84fc62452 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:37.634805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1a9e2-7cd4-4e65-8acc-8ab84fc62452 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:37.635334 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1a9e2-7cd4-4e65-8acc-8ab84fc62452 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:37.647835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1a9e2-7cd4-4e65-8acc-8ab84fc62452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-83b1a9e2-7cd4-4e65-8acc-8ab84fc62452 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:37.654386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:37.656376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:37.658204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:37.662722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 261/503] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:37.663756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1a9e2-7cd4-4e65-8acc-8ab84fc62452 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:37.674453 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:37.712314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc44434-6482-4731-9c69-df8ebd502d74 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:37.738578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:37.751016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c31e4-adf6-42e7-82ea-8fcb66ddde59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:37.751626 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c31e4-adf6-42e7-82ea-8fcb66ddde59 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:37.751937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c31e4-adf6-42e7-82ea-8fcb66ddde59 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:37.752221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c31e4-adf6-42e7-82ea-8fcb66ddde59 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:37.752858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c31e4-adf6-42e7-82ea-8fcb66ddde59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:37.753167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c31e4-adf6-42e7-82ea-8fcb66ddde59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:37.754556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c31e4-adf6-42e7-82ea-8fcb66ddde59 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:37.766616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 243/504] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:37.767436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c31e4-adf6-42e7-82ea-8fcb66ddde59 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:37.787892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1a9e2-7cd4-4e65-8acc-8ab84fc62452 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:37.827103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8db718-ab26-4670-8546-7af1352ae2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:37.827723 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8db718-ab26-4670-8546-7af1352ae2d9 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:37.828035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8db718-ab26-4670-8546-7af1352ae2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:37.828308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8db718-ab26-4670-8546-7af1352ae2d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:37.836132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8db718-ab26-4670-8546-7af1352ae2d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:37.837614 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8db718-ab26-4670-8546-7af1352ae2d9 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-1a8db718-ab26-4670-8546-7af1352ae2d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:37.850342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:37.851382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:37.852330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:37.852330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:37.852330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:37.852330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:37.852330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:37.852330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:37.852330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:37.852330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:37.852330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:31:37.852330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:37.852330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 262/505] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Aug 26 15:31:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:37.853434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8db718-ab26-4670-8546-7af1352ae2d9 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:37.880187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c31e4-adf6-42e7-82ea-8fcb66ddde59 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:37.911032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:37.917147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4af84-4618-4db8-ad78-0a76b8a37a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:37.917760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4af84-4618-4db8-ad78-0a76b8a37a11 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:37.918045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4af84-4618-4db8-ad78-0a76b8a37a11 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:37.918325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4af84-4618-4db8-ad78-0a76b8a37a11 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:37.919061 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4af84-4618-4db8-ad78-0a76b8a37a11 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:37.919371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4af84-4618-4db8-ad78-0a76b8a37a11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:37.920879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4af84-4618-4db8-ad78-0a76b8a37a11 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:37.959135 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 244/506] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Aug 26 15:31:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:37.959532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8db718-ab26-4670-8546-7af1352ae2d9 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:37.960005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4af84-4618-4db8-ad78-0a76b8a37a11 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:37.983564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:37.999298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddf217-90f6-43bc-89dd-67418718ac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:37.999935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddf217-90f6-43bc-89dd-67418718ac27 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:38.000226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddf217-90f6-43bc-89dd-67418718ac27 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:38.006907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddf217-90f6-43bc-89dd-67418718ac27 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:38.006907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddf217-90f6-43bc-89dd-67418718ac27 None admin] RBAC: Adding query filter params (name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:38.006907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddf217-90f6-43bc-89dd-67418718ac27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:38.006907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddf217-90f6-43bc-89dd-67418718ac27 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:38.011300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 263/507] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:38.012127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddf217-90f6-43bc-89dd-67418718ac27 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:38.062925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4af84-4618-4db8-ad78-0a76b8a37a11 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:38.097326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d1fc1-18ab-422b-88f1-fc50d0744e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:38.097960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d1fc1-18ab-422b-88f1-fc50d0744e81 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:38.098245 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d1fc1-18ab-422b-88f1-fc50d0744e81 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:38.098524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d1fc1-18ab-422b-88f1-fc50d0744e81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:38.109297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d1fc1-18ab-422b-88f1-fc50d0744e81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:38.110820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d1fc1-18ab-422b-88f1-fc50d0744e81 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-e66d1fc1-18ab-422b-88f1-fc50d0744e81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:38.126509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:38.128396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:38.130127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:38.130127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:38.130127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:38.130127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 245/508] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:38.130127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d1fc1-18ab-422b-88f1-fc50d0744e81 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:38.138900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddf217-90f6-43bc-89dd-67418718ac27 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:38.173392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:38.181949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0084f9-8eee-4723-984a-5a07c0d8f514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:38.182782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0084f9-8eee-4723-984a-5a07c0d8f514 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:38.183097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0084f9-8eee-4723-984a-5a07c0d8f514 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:38.183406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0084f9-8eee-4723-984a-5a07c0d8f514 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:38.193164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0084f9-8eee-4723-984a-5a07c0d8f514 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:38.195552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0084f9-8eee-4723-984a-5a07c0d8f514 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-8c0084f9-8eee-4723-984a-5a07c0d8f514 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:38.211128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:38.218727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 264/509] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:38.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0084f9-8eee-4723-984a-5a07c0d8f514 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:38.236168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d1fc1-18ab-422b-88f1-fc50d0744e81 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:38.264823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:38.270614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c5f1d-2b81-42db-903d-f38b5a96717d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:38.271219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c5f1d-2b81-42db-903d-f38b5a96717d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:38.271502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c5f1d-2b81-42db-903d-f38b5a96717d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:38.271781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c5f1d-2b81-42db-903d-f38b5a96717d None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:38.272591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c5f1d-2b81-42db-903d-f38b5a96717d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:38.272905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c5f1d-2b81-42db-903d-f38b5a96717d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:38.274345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c5f1d-2b81-42db-903d-f38b5a96717d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:38.292304 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 246/510] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Aug 26 15:31:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:38.293124 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c5f1d-2b81-42db-903d-f38b5a96717d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:38.326016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0084f9-8eee-4723-984a-5a07c0d8f514 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:38.360299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ea225-f91e-4c4f-861c-09a1ff06f961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:38.361064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ea225-f91e-4c4f-861c-09a1ff06f961 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:38.361433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ea225-f91e-4c4f-861c-09a1ff06f961 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:38.361812 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ea225-f91e-4c4f-861c-09a1ff06f961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:38.395575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:38.399452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c5f1d-2b81-42db-903d-f38b5a96717d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:38.444000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f996f35-38e8-45a8-8bc3-c51be7d7d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:38.444754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f996f35-38e8-45a8-8bc3-c51be7d7d338 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:38.445054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ea225-f91e-4c4f-861c-09a1ff06f961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:38.445221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f996f35-38e8-45a8-8bc3-c51be7d7d338 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:38.445544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f996f35-38e8-45a8-8bc3-c51be7d7d338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:38.447140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ea225-f91e-4c4f-861c-09a1ff06f961 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:38.457287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f996f35-38e8-45a8-8bc3-c51be7d7d338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:38.458755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f996f35-38e8-45a8-8bc3-c51be7d7d338 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-7f996f35-38e8-45a8-8bc3-c51be7d7d338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:38.473770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:38.475602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:38.477565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:38.477565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:38.477565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:38.477565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 247/511] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:38.477565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f996f35-38e8-45a8-8bc3-c51be7d7d338 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:38.498539 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:38.566287 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 265/512] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:38.566287 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ea225-f91e-4c4f-861c-09a1ff06f961 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:38.597286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f996f35-38e8-45a8-8bc3-c51be7d7d338 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:38.635119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f12c-fa87-45ae-b20c-8e1bab6111bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:38.635738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f12c-fa87-45ae-b20c-8e1bab6111bc None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:38.636053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f12c-fa87-45ae-b20c-8e1bab6111bc None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:38.636334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f12c-fa87-45ae-b20c-8e1bab6111bc None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:38.636988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8f12c-fa87-45ae-b20c-8e1bab6111bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:38.637288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8f12c-fa87-45ae-b20c-8e1bab6111bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:38.638683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8f12c-fa87-45ae-b20c-8e1bab6111bc None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:38.651508 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 248/513] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:38.652318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f12c-fa87-45ae-b20c-8e1bab6111bc None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:38.665740 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:38.670792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ea225-f91e-4c4f-861c-09a1ff06f961 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:38.714185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce8e11-8f49-4dbe-a5b2-9bd98bb9ef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:38.715063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce8e11-8f49-4dbe-a5b2-9bd98bb9ef75 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:38.715520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce8e11-8f49-4dbe-a5b2-9bd98bb9ef75 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:38.716011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce8e11-8f49-4dbe-a5b2-9bd98bb9ef75 None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:38.717242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce8e11-8f49-4dbe-a5b2-9bd98bb9ef75 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:38.717749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce8e11-8f49-4dbe-a5b2-9bd98bb9ef75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:38.720070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce8e11-8f49-4dbe-a5b2-9bd98bb9ef75 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:38.760659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:38.765550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 266/514] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Aug 26 15:31:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:38.766496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce8e11-8f49-4dbe-a5b2-9bd98bb9ef75 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:38.774337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f12c-fa87-45ae-b20c-8e1bab6111bc None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:38.813633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c5f7c-fc0e-469a-a36f-ab04c4fbee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:38.814341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c5f7c-fc0e-469a-a36f-ab04c4fbee13 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:38.814802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c5f7c-fc0e-469a-a36f-ab04c4fbee13 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:38.815169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c5f7c-fc0e-469a-a36f-ab04c4fbee13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:38.882039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce8e11-8f49-4dbe-a5b2-9bd98bb9ef75 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:38.885174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c5f7c-fc0e-469a-a36f-ab04c4fbee13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:38.886674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c5f7c-fc0e-469a-a36f-ab04c4fbee13 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:38.897832 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:38.919188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4ee83-e02b-47d3-a4c0-aedc37656042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:38.919721 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4ee83-e02b-47d3-a4c0-aedc37656042 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:38.920029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4ee83-e02b-47d3-a4c0-aedc37656042 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:38.920311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4ee83-e02b-47d3-a4c0-aedc37656042 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:38.920962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4ee83-e02b-47d3-a4c0-aedc37656042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:38.921258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4ee83-e02b-47d3-a4c0-aedc37656042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:38.923183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4ee83-e02b-47d3-a4c0-aedc37656042 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:38.943536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 267/515] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:38.945020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4ee83-e02b-47d3-a4c0-aedc37656042 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:38.994439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 249/516] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:38.995348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c5f7c-fc0e-469a-a36f-ab04c4fbee13 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:39.017825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:39.081389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4ee83-e02b-47d3-a4c0-aedc37656042 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:39.125984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0860ca-0a92-4a8b-bef3-ce599f4f6a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:39.126607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0860ca-0a92-4a8b-bef3-ce599f4f6a1b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:39.126886 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0860ca-0a92-4a8b-bef3-ce599f4f6a1b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:39.127164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0860ca-0a92-4a8b-bef3-ce599f4f6a1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:39.132756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c5f7c-fc0e-469a-a36f-ab04c4fbee13 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:39.137866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0860ca-0a92-4a8b-bef3-ce599f4f6a1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:39.139320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0860ca-0a92-4a8b-bef3-ce599f4f6a1b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:39.143645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-ef0860ca-0a92-4a8b-bef3-ce599f4f6a1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:39.153837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:39.155582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:39.157474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:39.157474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:39.157474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:39.157474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 268/517] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:39.157474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0860ca-0a92-4a8b-bef3-ce599f4f6a1b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:39.171945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04241829-9365-470f-970c-28ec777abd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:39.172489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04241829-9365-470f-970c-28ec777abd46 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:39.172767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04241829-9365-470f-970c-28ec777abd46 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:39.173040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04241829-9365-470f-970c-28ec777abd46 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:39.173675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04241829-9365-470f-970c-28ec777abd46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:39.175100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04241829-9365-470f-970c-28ec777abd46 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:39.237638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 250/518] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:31:38 2022] POST /identity/v3/users => generated 247 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:31:39.238462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04241829-9365-470f-970c-28ec777abd46 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:39.271833 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0860ca-0a92-4a8b-bef3-ce599f4f6a1b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:39.291008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:39.321352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7cd2b-efbb-48ae-9b31-823e9d164a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:39.322382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7cd2b-efbb-48ae-9b31-823e9d164a0f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:39.322885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7cd2b-efbb-48ae-9b31-823e9d164a0f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:39.323491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7cd2b-efbb-48ae-9b31-823e9d164a0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:39.369691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04241829-9365-470f-970c-28ec777abd46 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:39.405785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7cd2b-efbb-48ae-9b31-823e9d164a0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:39.407265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7cd2b-efbb-48ae-9b31-823e9d164a0f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:39.408084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e4f08-e654-478f-8458-9a38b8d6214c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:39.408797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e4f08-e654-478f-8458-9a38b8d6214c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:39.409155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e4f08-e654-478f-8458-9a38b8d6214c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:39.409498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e4f08-e654-478f-8458-9a38b8d6214c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:39.421514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e4f08-e654-478f-8458-9a38b8d6214c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:39.423069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e4f08-e654-478f-8458-9a38b8d6214c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-5e8e4f08-e654-478f-8458-9a38b8d6214c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:39.437475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:39.439709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 251/519] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:39.440841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e4f08-e654-478f-8458-9a38b8d6214c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:39.472277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:39.532816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 269/520] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:39.532816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7cd2b-efbb-48ae-9b31-823e9d164a0f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:39.544037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:39.555915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e4f08-e654-478f-8458-9a38b8d6214c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:39.594617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f9ad9-9fd4-4294-8746-d2adbd8321b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:39.595341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f9ad9-9fd4-4294-8746-d2adbd8321b8 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:39.595693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f9ad9-9fd4-4294-8746-d2adbd8321b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:39.596163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f9ad9-9fd4-4294-8746-d2adbd8321b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:39.610325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f9ad9-9fd4-4294-8746-d2adbd8321b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:39.611812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f9ad9-9fd4-4294-8746-d2adbd8321b8 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-986f9ad9-9fd4-4294-8746-d2adbd8321b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:39.628238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:39.629195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:39.630201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:39.630201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:39.630201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:39.631085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 252/521] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:39.631991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f9ad9-9fd4-4294-8746-d2adbd8321b8 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:39.643132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7cd2b-efbb-48ae-9b31-823e9d164a0f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:39.682213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a781ce1-ed52-4392-8dd3-0b5ccdd78f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:39.682848 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a781ce1-ed52-4392-8dd3-0b5ccdd78f37 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:39.683240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a781ce1-ed52-4392-8dd3-0b5ccdd78f37 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:39.683355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a781ce1-ed52-4392-8dd3-0b5ccdd78f37 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:39.684224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a781ce1-ed52-4392-8dd3-0b5ccdd78f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:39.684328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a781ce1-ed52-4392-8dd3-0b5ccdd78f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:39.685688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a781ce1-ed52-4392-8dd3-0b5ccdd78f37 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:39.702081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 270/522] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:39.702081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a781ce1-ed52-4392-8dd3-0b5ccdd78f37 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:39.720800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:39.742559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f9ad9-9fd4-4294-8746-d2adbd8321b8 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:39.778861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222a9a0-01cb-4255-bf93-3277068adebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:39.779474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a9a0-01cb-4255-bf93-3277068adebd None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:39.779757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a9a0-01cb-4255-bf93-3277068adebd None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:39.780062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a9a0-01cb-4255-bf93-3277068adebd None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:39.780800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222a9a0-01cb-4255-bf93-3277068adebd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:39.781108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222a9a0-01cb-4255-bf93-3277068adebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:39.782583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222a9a0-01cb-4255-bf93-3277068adebd None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:39.798659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:39.804351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 253/523] 10.222.0.98 () {56 vars in 1139 bytes} [Fri Aug 26 15:31:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:39.804351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222a9a0-01cb-4255-bf93-3277068adebd None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:39.833311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a781ce1-ed52-4392-8dd3-0b5ccdd78f37 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:39.868772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde40945-21d5-4864-8c47-920f50dae5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:39.869407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde40945-21d5-4864-8c47-920f50dae5f7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:39.869684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde40945-21d5-4864-8c47-920f50dae5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:39.869956 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde40945-21d5-4864-8c47-920f50dae5f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:39.882335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde40945-21d5-4864-8c47-920f50dae5f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:39.884314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde40945-21d5-4864-8c47-920f50dae5f7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-cde40945-21d5-4864-8c47-920f50dae5f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:39.898803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:39.900517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:39.901404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:39.901404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:39.901404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:39.901404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 271/524] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:39.902189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde40945-21d5-4864-8c47-920f50dae5f7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:39.916535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222a9a0-01cb-4255-bf93-3277068adebd None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:39.955415 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2a49a-a4f3-4ef0-a5b7-a580c714be8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:39.955964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2a49a-a4f3-4ef0-a5b7-a580c714be8c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:39.956248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2a49a-a4f3-4ef0-a5b7-a580c714be8c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:39.956522 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2a49a-a4f3-4ef0-a5b7-a580c714be8c None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:39.957172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2a49a-a4f3-4ef0-a5b7-a580c714be8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:39.957481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2a49a-a4f3-4ef0-a5b7-a580c714be8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:39.958862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2a49a-a4f3-4ef0-a5b7-a580c714be8c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:39.967194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:39.973073 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 254/525] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:39.973978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2a49a-a4f3-4ef0-a5b7-a580c714be8c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:40.011805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde40945-21d5-4864-8c47-920f50dae5f7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:40.036812 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:40.047233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27334548-ed2a-4ff9-ad36-9c278a1f1459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:40.048007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27334548-ed2a-4ff9-ad36-9c278a1f1459 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:40.048360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27334548-ed2a-4ff9-ad36-9c278a1f1459 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:40.048705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27334548-ed2a-4ff9-ad36-9c278a1f1459 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:40.056260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27334548-ed2a-4ff9-ad36-9c278a1f1459 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:40.057828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27334548-ed2a-4ff9-ad36-9c278a1f1459 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-27334548-ed2a-4ff9-ad36-9c278a1f1459 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:40.069693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:40.070756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:40.072399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:40.072399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:40.072399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:40.072399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:40.072399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:40.072399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:40.072399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:40.072399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:40.072856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 272/526] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:40.073862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27334548-ed2a-4ff9-ad36-9c278a1f1459 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:40.083260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2a49a-a4f3-4ef0-a5b7-a580c714be8c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:40.117798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c18a19-655e-453e-ba7e-f8166a5f1272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:40.119065 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c18a19-655e-453e-ba7e-f8166a5f1272 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:40.119442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c18a19-655e-453e-ba7e-f8166a5f1272 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:40.119831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c18a19-655e-453e-ba7e-f8166a5f1272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:40.180192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27334548-ed2a-4ff9-ad36-9c278a1f1459 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:40.185060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:40.191837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c18a19-655e-453e-ba7e-f8166a5f1272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:40.193478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c18a19-655e-453e-ba7e-f8166a5f1272 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:40.220304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d88051-da36-485f-88b2-def2611e4c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:40.220304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d88051-da36-485f-88b2-def2611e4c8a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:40.220304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d88051-da36-485f-88b2-def2611e4c8a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:40.220304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d88051-da36-485f-88b2-def2611e4c8a None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:40.220739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d88051-da36-485f-88b2-def2611e4c8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:40.220739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d88051-da36-485f-88b2-def2611e4c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:40.220739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d88051-da36-485f-88b2-def2611e4c8a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:40.231972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 273/527] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:40.233003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d88051-da36-485f-88b2-def2611e4c8a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:40.281869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:40.318703 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 255/528] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:40.318703 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c18a19-655e-453e-ba7e-f8166a5f1272 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:40.337512 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d88051-da36-485f-88b2-def2611e4c8a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:40.374262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a9f5f-ae04-4439-a21f-af45ce4d8867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:40.374900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9f5f-ae04-4439-a21f-af45ce4d8867 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:40.375190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9f5f-ae04-4439-a21f-af45ce4d8867 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:40.375470 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9f5f-ae04-4439-a21f-af45ce4d8867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:40.382177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a9f5f-ae04-4439-a21f-af45ce4d8867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:40.383805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a9f5f-ae04-4439-a21f-af45ce4d8867 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-bf8a9f5f-ae04-4439-a21f-af45ce4d8867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:40.395193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:40.396247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:40.397164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:40.397164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:40.397164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:40.397164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:40.397164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:40.397164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:40.397164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:40.397164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:40.397164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 274/529] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:40.397164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a9f5f-ae04-4439-a21f-af45ce4d8867 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:40.430123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:40.437149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c18a19-655e-453e-ba7e-f8166a5f1272 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:40.473416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ef492-bd8d-4c56-b5ab-0da551f7cc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:40.474076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ef492-bd8d-4c56-b5ab-0da551f7cc17 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:40.474419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ef492-bd8d-4c56-b5ab-0da551f7cc17 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:40.474985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ef492-bd8d-4c56-b5ab-0da551f7cc17 None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:40.476661 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ef492-bd8d-4c56-b5ab-0da551f7cc17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:40.477045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ef492-bd8d-4c56-b5ab-0da551f7cc17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:40.479074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ef492-bd8d-4c56-b5ab-0da551f7cc17 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:40.498664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 256/530] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:40.499636 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ef492-bd8d-4c56-b5ab-0da551f7cc17 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:40.514044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a9f5f-ae04-4439-a21f-af45ce4d8867 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:40.516672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:40.548935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3e838-e380-4d0c-b0be-0df394ecff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:40.549645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3e838-e380-4d0c-b0be-0df394ecff91 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:40.549999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3e838-e380-4d0c-b0be-0df394ecff91 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:40.550346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3e838-e380-4d0c-b0be-0df394ecff91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:40.558230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3e838-e380-4d0c-b0be-0df394ecff91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:40.560360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3e838-e380-4d0c-b0be-0df394ecff91 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-1eb3e838-e380-4d0c-b0be-0df394ecff91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:40.572731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:40.573880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:40.575477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:40.575477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:40.575477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:40.575477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:40.575477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:40.575477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:40.575477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:40.575477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:40.575939 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 275/531] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:40.577331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3e838-e380-4d0c-b0be-0df394ecff91 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:40.624396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ef492-bd8d-4c56-b5ab-0da551f7cc17 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:40.663438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243dbec3-66ae-4b40-a0a9-9b624a467f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:40.664109 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243dbec3-66ae-4b40-a0a9-9b624a467f80 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:40.664399 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243dbec3-66ae-4b40-a0a9-9b624a467f80 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:40.664711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243dbec3-66ae-4b40-a0a9-9b624a467f80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:40.676123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243dbec3-66ae-4b40-a0a9-9b624a467f80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:40.677587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243dbec3-66ae-4b40-a0a9-9b624a467f80 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:40.692508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3e838-e380-4d0c-b0be-0df394ecff91 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-243dbec3-66ae-4b40-a0a9-9b624a467f80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:40.694621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:40.695566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:40.698210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:40.698210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:40.698210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:40.698210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 257/532] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:40.698210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243dbec3-66ae-4b40-a0a9-9b624a467f80 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:40.711072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:40.737992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85022f82-c191-41af-8b27-75aa61fb292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:40.738523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85022f82-c191-41af-8b27-75aa61fb292b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:40.738812 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85022f82-c191-41af-8b27-75aa61fb292b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:40.739089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85022f82-c191-41af-8b27-75aa61fb292b None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:40.739846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85022f82-c191-41af-8b27-75aa61fb292b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:40.740190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85022f82-c191-41af-8b27-75aa61fb292b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:40.741626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85022f82-c191-41af-8b27-75aa61fb292b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:40.760976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 276/533] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:40.766437 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85022f82-c191-41af-8b27-75aa61fb292b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:40.793447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:40.807949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243dbec3-66ae-4b40-a0a9-9b624a467f80 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:40.847343 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600947a-6aec-43ea-8118-d94b7f7cd37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:40.847972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600947a-6aec-43ea-8118-d94b7f7cd37c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:40.848669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600947a-6aec-43ea-8118-d94b7f7cd37c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:40.848963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600947a-6aec-43ea-8118-d94b7f7cd37c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:40.849698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600947a-6aec-43ea-8118-d94b7f7cd37c None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=cef193a54eff44f2b53716e3ec1a29cf) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:40.850009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600947a-6aec-43ea-8118-d94b7f7cd37c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:40.852137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600947a-6aec-43ea-8118-d94b7f7cd37c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:40.859009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 258/534] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:40 2022] GET /identity/v3/role_assignments?user.id=cef193a54eff44f2b53716e3ec1a29cf&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:40.859799 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600947a-6aec-43ea-8118-d94b7f7cd37c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:40.891443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85022f82-c191-41af-8b27-75aa61fb292b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:40.936557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c58b-3e7c-41c9-801b-3aa9840e3126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:40.937541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c58b-3e7c-41c9-801b-3aa9840e3126 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:40.938149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c58b-3e7c-41c9-801b-3aa9840e3126 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:40.938491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c58b-3e7c-41c9-801b-3aa9840e3126 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:40.939286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8c58b-3e7c-41c9-801b-3aa9840e3126 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:40.939674 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8c58b-3e7c-41c9-801b-3aa9840e3126 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:40.941257 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8c58b-3e7c-41c9-801b-3aa9840e3126 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:40.960622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 277/535] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:40.960622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c58b-3e7c-41c9-801b-3aa9840e3126 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:40.967654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:41.008412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600947a-6aec-43ea-8118-d94b7f7cd37c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:41.049589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37585847-6b1e-4871-b0c4-b1d4d1ed9331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:41.050315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37585847-6b1e-4871-b0c4-b1d4d1ed9331 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:41.050712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37585847-6b1e-4871-b0c4-b1d4d1ed9331 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:41.051014 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37585847-6b1e-4871-b0c4-b1d4d1ed9331 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:41.051754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37585847-6b1e-4871-b0c4-b1d4d1ed9331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:41.052158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37585847-6b1e-4871-b0c4-b1d4d1ed9331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:41.054423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37585847-6b1e-4871-b0c4-b1d4d1ed9331 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:41.068027 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 259/536] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:41.069530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37585847-6b1e-4871-b0c4-b1d4d1ed9331 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:41.072775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:41.093321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c58b-3e7c-41c9-801b-3aa9840e3126 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:41.140820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc758b9-57fe-4097-b8a0-aee010454ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:41.141671 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc758b9-57fe-4097-b8a0-aee010454ef9 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:41.142096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc758b9-57fe-4097-b8a0-aee010454ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:41.142514 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc758b9-57fe-4097-b8a0-aee010454ef9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:41.143545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc758b9-57fe-4097-b8a0-aee010454ef9 None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=d1b01c21c38949938141eefdc4939385) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:41.144053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc758b9-57fe-4097-b8a0-aee010454ef9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:41.146454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc758b9-57fe-4097-b8a0-aee010454ef9 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:41.153581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 278/537] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:40 2022] GET /identity/v3/role_assignments?user.id=d1b01c21c38949938141eefdc4939385&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:41.155038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc758b9-57fe-4097-b8a0-aee010454ef9 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:41.188059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37585847-6b1e-4871-b0c4-b1d4d1ed9331 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:41.230581 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:41.237858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cf6bb-3e3d-4a75-809d-b6df0d7b5f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:41.239013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cf6bb-3e3d-4a75-809d-b6df0d7b5f7e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:41.239444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cf6bb-3e3d-4a75-809d-b6df0d7b5f7e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:41.246234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cf6bb-3e3d-4a75-809d-b6df0d7b5f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:41.246234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cf6bb-3e3d-4a75-809d-b6df0d7b5f7e None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=3a1ff8aff9e048c6972843e0b50c4d45) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:41.246234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cf6bb-3e3d-4a75-809d-b6df0d7b5f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:41.246234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cf6bb-3e3d-4a75-809d-b6df0d7b5f7e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:41.252132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 260/538] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:41 2022] GET /identity/v3/role_assignments?user.id=3a1ff8aff9e048c6972843e0b50c4d45&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:41.288910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc758b9-57fe-4097-b8a0-aee010454ef9 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:41.331823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46257c-3f88-4324-ab68-0e9de4c81ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:41.332506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46257c-3f88-4324-ab68-0e9de4c81ae8 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:41.332795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46257c-3f88-4324-ab68-0e9de4c81ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:41.333073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46257c-3f88-4324-ab68-0e9de4c81ae8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:41.341682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46257c-3f88-4324-ab68-0e9de4c81ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:41.343190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46257c-3f88-4324-ab68-0e9de4c81ae8 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-9a46257c-3f88-4324-ab68-0e9de4c81ae8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:41.357383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:41.358311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:41.359185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:41.359185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:41.359185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:41.359185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:41.359185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:41.359185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:41.359185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:41.359185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:41.359185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 279/539] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:41.364107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46257c-3f88-4324-ab68-0e9de4c81ae8 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:41.364341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b41da-dd4c-4ff9-aa88-33cbd49c3a0d None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:41.364637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b41da-dd4c-4ff9-aa88-33cbd49c3a0d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:41.364920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b41da-dd4c-4ff9-aa88-33cbd49c3a0d None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:41.366620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 261/540] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:41.373382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5175a33-cde4-4294-a7fb-b595fe498e44 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:41.373692 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5175a33-cde4-4294-a7fb-b595fe498e44 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:41.373942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5175a33-cde4-4294-a7fb-b595fe498e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:41.390671 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:41.398187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c5175a33-cde4-4294-a7fb-b595fe498e44 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:41.399919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c5175a33-cde4-4294-a7fb-b595fe498e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:41.487756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 262/541] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:41.517285 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46257c-3f88-4324-ab68-0e9de4c81ae8 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:41.553164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f5fc6-e5be-4b45-8015-7383e2777417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:41.553733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f5fc6-e5be-4b45-8015-7383e2777417 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:41.554018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f5fc6-e5be-4b45-8015-7383e2777417 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:41.554296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f5fc6-e5be-4b45-8015-7383e2777417 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:41.555302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f5fc6-e5be-4b45-8015-7383e2777417 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:41.555777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f5fc6-e5be-4b45-8015-7383e2777417 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:41.557389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f5fc6-e5be-4b45-8015-7383e2777417 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:41.570468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 280/542] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:41.576404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5175a33-cde4-4294-a7fb-b595fe498e44 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:41.700730 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5175a33-cde4-4294-a7fb-b595fe498e44 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:41.733824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:41.747599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e873c4-054c-4471-b8c3-c0426ad81060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:41.748292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e873c4-054c-4471-b8c3-c0426ad81060 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:41.748637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e873c4-054c-4471-b8c3-c0426ad81060 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:41.748912 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e873c4-054c-4471-b8c3-c0426ad81060 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:41.749640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e873c4-054c-4471-b8c3-c0426ad81060 None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=8c6cad18359143868276eb9d79a3fd3b) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:41.749945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e873c4-054c-4471-b8c3-c0426ad81060 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:41.751375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e873c4-054c-4471-b8c3-c0426ad81060 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:41.757084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 263/543] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:41 2022] GET /identity/v3/role_assignments?user.id=8c6cad18359143868276eb9d79a3fd3b&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:41.844598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f5fc6-e5be-4b45-8015-7383e2777417 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:41.946386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:41.982332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f5fc6-e5be-4b45-8015-7383e2777417 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:42.018358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f07c5d-27fb-4b44-9753-b34959c41f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:42.019000 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07c5d-27fb-4b44-9753-b34959c41f41 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:42.019283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07c5d-27fb-4b44-9753-b34959c41f41 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:42.019563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07c5d-27fb-4b44-9753-b34959c41f41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:42.031028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f07c5d-27fb-4b44-9753-b34959c41f41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-d0f07c5d-27fb-4b44-9753-b34959c41f41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:42.036226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:42.038276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:42.040265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 281/544] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:42.046807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e873c4-054c-4471-b8c3-c0426ad81060 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:42.184093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e873c4-054c-4471-b8c3-c0426ad81060 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:42.233266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace5760-5fdf-43c3-abd3-ea0dfc322721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:42.233893 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5760-5fdf-43c3-abd3-ea0dfc322721 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:42.234175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5760-5fdf-43c3-abd3-ea0dfc322721 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:42.234456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5760-5fdf-43c3-abd3-ea0dfc322721 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:42.235133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5760-5fdf-43c3-abd3-ea0dfc322721 None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:42.235441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5760-5fdf-43c3-abd3-ea0dfc322721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:42.236850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5760-5fdf-43c3-abd3-ea0dfc322721 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:42.252045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 264/545] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:42.257559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f07c5d-27fb-4b44-9753-b34959c41f41 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:42.385705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f07c5d-27fb-4b44-9753-b34959c41f41 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:42.424884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:42.433637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48369cf3-36bb-49ef-ab47-6e5a637e5e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:42.441213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48369cf3-36bb-49ef-ab47-6e5a637e5e9e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:42.441213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48369cf3-36bb-49ef-ab47-6e5a637e5e9e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:42.441213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48369cf3-36bb-49ef-ab47-6e5a637e5e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:42.450501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48369cf3-36bb-49ef-ab47-6e5a637e5e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:42.451991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48369cf3-36bb-49ef-ab47-6e5a637e5e9e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-48369cf3-36bb-49ef-ab47-6e5a637e5e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:42.467489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:42.471283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:42.472863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:42.472863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:42.472863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:42.472863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 282/546] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:42.475257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace5760-5fdf-43c3-abd3-ea0dfc322721 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:42.546210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:42.611763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace5760-5fdf-43c3-abd3-ea0dfc322721 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:42.647007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87da32a-c9a1-420d-bf3a-61ee8334adc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:42.647675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87da32a-c9a1-420d-bf3a-61ee8334adc1 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:42.647983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87da32a-c9a1-420d-bf3a-61ee8334adc1 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:42.648271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87da32a-c9a1-420d-bf3a-61ee8334adc1 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:42.649478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87da32a-c9a1-420d-bf3a-61ee8334adc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:42.649787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87da32a-c9a1-420d-bf3a-61ee8334adc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:42.651161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87da32a-c9a1-420d-bf3a-61ee8334adc1 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:42.663300 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 265/547] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:42.668967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48369cf3-36bb-49ef-ab47-6e5a637e5e9e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:42.782918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48369cf3-36bb-49ef-ab47-6e5a637e5e9e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:42.828231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276916e-d26d-4b4c-9bb0-7b34d4c6c62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:42.828869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276916e-d26d-4b4c-9bb0-7b34d4c6c62f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:42.829209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276916e-d26d-4b4c-9bb0-7b34d4c6c62f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:42.829472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276916e-d26d-4b4c-9bb0-7b34d4c6c62f None admin] PATH_INFO: `/v3/users/swift` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:42.838122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276916e-d26d-4b4c-9bb0-7b34d4c6c62f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:42.839577 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276916e-d26d-4b4c-9bb0-7b34d4c6c62f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-c276916e-d26d-4b4c-9bb0-7b34d4c6c62f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:42.851022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:42.853927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:42.856191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:42.856191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:42.856191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:42.856191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:42.856191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:42.856191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:42.856191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:42.856191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:42.856191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:31:42.856191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:42.856191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 283/548] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:31:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:42.859889 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87da32a-c9a1-420d-bf3a-61ee8334adc1 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:42.963888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87da32a-c9a1-420d-bf3a-61ee8334adc1 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:42.978686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:43.007706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d968e6-66ca-44c7-90de-adb7d5e1aea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:43.009042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d968e6-66ca-44c7-90de-adb7d5e1aea7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:43.009330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d968e6-66ca-44c7-90de-adb7d5e1aea7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:43.009601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d968e6-66ca-44c7-90de-adb7d5e1aea7 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:43.010344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d968e6-66ca-44c7-90de-adb7d5e1aea7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:43.010701 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d968e6-66ca-44c7-90de-adb7d5e1aea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:43.012162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d968e6-66ca-44c7-90de-adb7d5e1aea7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:43.035385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 266/549] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Aug 26 15:31:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:43.040978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276916e-d26d-4b4c-9bb0-7b34d4c6c62f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:43.095031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:43.165286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0665dc-b986-4c89-a4c3-902454bdf205 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:43.165587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0665dc-b986-4c89-a4c3-902454bdf205 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:43.165962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0665dc-b986-4c89-a4c3-902454bdf205 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:43.167518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 267/550] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:43.172421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276916e-d26d-4b4c-9bb0-7b34d4c6c62f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:43.172544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db239ad-6b9e-4d86-adbb-2f1fde6004f3 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:43.172734 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db239ad-6b9e-4d86-adbb-2f1fde6004f3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:43.172998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db239ad-6b9e-4d86-adbb-2f1fde6004f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:43.209640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-9db239ad-6b9e-4d86-adbb-2f1fde6004f3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:43.210858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9db239ad-6b9e-4d86-adbb-2f1fde6004f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:43.231457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495de793-5d9e-4404-af7d-f78cb48e8348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:43.233398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495de793-5d9e-4404-af7d-f78cb48e8348 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:43.233398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495de793-5d9e-4404-af7d-f78cb48e8348 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:43.233398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495de793-5d9e-4404-af7d-f78cb48e8348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:43.295145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 268/551] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:43.350408 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495de793-5d9e-4404-af7d-f78cb48e8348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:43.352525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495de793-5d9e-4404-af7d-f78cb48e8348 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:43.394558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:43.455146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 284/552] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 15:31:43.460554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db239ad-6b9e-4d86-adbb-2f1fde6004f3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:43.506400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a4167-fb8c-4dd3-b751-9d11cf45a069 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:43.506748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a4167-fb8c-4dd3-b751-9d11cf45a069 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:43.506963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a4167-fb8c-4dd3-b751-9d11cf45a069 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:43.508627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 285/553] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:43.516884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6a041-3013-4e50-9c67-7d4a5a831671 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:43.517173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6a041-3013-4e50-9c67-7d4a5a831671 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:43.517891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6a041-3013-4e50-9c67-7d4a5a831671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:43.539806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-41d6a041-3013-4e50-9c67-7d4a5a831671 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:43.541050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-41d6a041-3013-4e50-9c67-7d4a5a831671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:43.619313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 286/554] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:43.625085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6a041-3013-4e50-9c67-7d4a5a831671 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:43.634638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:43.637659 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db239ad-6b9e-4d86-adbb-2f1fde6004f3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:43.676517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535689a-b5d2-4807-9763-87c77a77ea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:43.677163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535689a-b5d2-4807-9763-87c77a77ea94 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:43.677447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535689a-b5d2-4807-9763-87c77a77ea94 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:43.677714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535689a-b5d2-4807-9763-87c77a77ea94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:43.700667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535689a-b5d2-4807-9763-87c77a77ea94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:43.702163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535689a-b5d2-4807-9763-87c77a77ea94 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-b535689a-b5d2-4807-9763-87c77a77ea94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:43.729405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:43.730408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:43.731301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:43.731301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:43.731301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:43.731301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 269/555] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:43.741805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535689a-b5d2-4807-9763-87c77a77ea94 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:43.763150 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:43.821891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6a041-3013-4e50-9c67-7d4a5a831671 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:43.861038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c4881-1507-4f83-ad92-5145cc4f6129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:43.861655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c4881-1507-4f83-ad92-5145cc4f6129 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:43.861899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c4881-1507-4f83-ad92-5145cc4f6129 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:43.862139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c4881-1507-4f83-ad92-5145cc4f6129 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:43.873053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c4881-1507-4f83-ad92-5145cc4f6129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-e79c4881-1507-4f83-ad92-5145cc4f6129 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:43.878311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:43.879226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:43.880182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 287/556] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:43.881395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e938bda-5e4a-4a9e-8146-f1e412236e55 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:43.881475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e938bda-5e4a-4a9e-8146-f1e412236e55 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:43.884004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e938bda-5e4a-4a9e-8146-f1e412236e55 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:43.887202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 288/557] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:43.888869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde50338-c3a1-4596-be22-c1b924098031 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:43.889306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde50338-c3a1-4596-be22-c1b924098031 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:43.889728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde50338-c3a1-4596-be22-c1b924098031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:43.903637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535689a-b5d2-4807-9763-87c77a77ea94 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:43.916867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-dde50338-c3a1-4596-be22-c1b924098031 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:43.918925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-dde50338-c3a1-4596-be22-c1b924098031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:43.947277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3b847-cc7c-42e8-83f5-70dee033483c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:43.947914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b847-cc7c-42e8-83f5-70dee033483c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:43.948206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b847-cc7c-42e8-83f5-70dee033483c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:43.948483 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b847-cc7c-42e8-83f5-70dee033483c None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:43.949130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3b847-cc7c-42e8-83f5-70dee033483c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:43.949434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3b847-cc7c-42e8-83f5-70dee033483c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:43.950814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3b847-cc7c-42e8-83f5-70dee033483c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:43.965130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 270/558] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:43.966060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3b847-cc7c-42e8-83f5-70dee033483c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:43.986113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:43.999228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 289/559] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:44.000108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde50338-c3a1-4596-be22-c1b924098031 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:44.082252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:44.090154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3b847-cc7c-42e8-83f5-70dee033483c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:44.128129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799300a-7599-4337-b754-94243f004925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:44.128719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799300a-7599-4337-b754-94243f004925 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.129011 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799300a-7599-4337-b754-94243f004925 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.129205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799300a-7599-4337-b754-94243f004925 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.129879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799300a-7599-4337-b754-94243f004925 None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:44.130183 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799300a-7599-4337-b754-94243f004925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:44.131660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799300a-7599-4337-b754-94243f004925 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:44.136801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde50338-c3a1-4596-be22-c1b924098031 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:44.142630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 271/560] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:44.143537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799300a-7599-4337-b754-94243f004925 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:44.187195 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36e567-7018-4f9b-995e-25bc59d4e78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:44.188804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36e567-7018-4f9b-995e-25bc59d4e78c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.189299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36e567-7018-4f9b-995e-25bc59d4e78c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.190288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36e567-7018-4f9b-995e-25bc59d4e78c None admin] PATH_INFO: `/v3/roles/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.206059 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36e567-7018-4f9b-995e-25bc59d4e78c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-db36e567-7018-4f9b-995e-25bc59d4e78c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:44.214535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:44.215494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:44.216487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:44.222505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 290/561] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:44.222505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36e567-7018-4f9b-995e-25bc59d4e78c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:44.234116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:44.262302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799300a-7599-4337-b754-94243f004925 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:44.311535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f905f-db18-4f30-b6fd-fc20078be9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:44.312640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f905f-db18-4f30-b6fd-fc20078be9f3 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.313068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f905f-db18-4f30-b6fd-fc20078be9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.313605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f905f-db18-4f30-b6fd-fc20078be9f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.322945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f905f-db18-4f30-b6fd-fc20078be9f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:44.325293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f905f-db18-4f30-b6fd-fc20078be9f3 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-e56f905f-db18-4f30-b6fd-fc20078be9f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:44.343165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:44.344512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:44.345412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:44.345412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:44.345412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:44.345412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:44.345412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:44.345412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:44.345412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:44.345412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:44.345412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 272/562] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:44.345412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f905f-db18-4f30-b6fd-fc20078be9f3 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:44.367946 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36e567-7018-4f9b-995e-25bc59d4e78c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:44.390095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:44.405504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b2658-5a3f-4084-9802-5b612e822f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:44.406129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2658-5a3f-4084-9802-5b612e822f29 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.406407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2658-5a3f-4084-9802-5b612e822f29 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.406679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2658-5a3f-4084-9802-5b612e822f29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.416922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b2658-5a3f-4084-9802-5b612e822f29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:44.418377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b2658-5a3f-4084-9802-5b612e822f29 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-743b2658-5a3f-4084-9802-5b612e822f29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:44.433100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:44.434186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:44.435097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:44.435097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:44.435097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:44.435097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 291/563] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:44.435097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b2658-5a3f-4084-9802-5b612e822f29 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:44.459086 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f905f-db18-4f30-b6fd-fc20078be9f3 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:44.498213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeb8b6-d39a-406e-82ea-70eb92012102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:44.498827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeb8b6-d39a-406e-82ea-70eb92012102 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.499226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeb8b6-d39a-406e-82ea-70eb92012102 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.499508 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeb8b6-d39a-406e-82ea-70eb92012102 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.500321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eeb8b6-d39a-406e-82ea-70eb92012102 None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:44.500632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eeb8b6-d39a-406e-82ea-70eb92012102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:44.502220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eeb8b6-d39a-406e-82ea-70eb92012102 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:44.503762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:44.517301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 273/564] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:44.518142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeb8b6-d39a-406e-82ea-70eb92012102 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:44.542449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b2658-5a3f-4084-9802-5b612e822f29 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:44.582792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbe44a-53de-483a-beb5-4a33c6065fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:44.583427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbe44a-53de-483a-beb5-4a33c6065fe0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.583714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbe44a-53de-483a-beb5-4a33c6065fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.584022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbe44a-53de-483a-beb5-4a33c6065fe0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.594188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbe44a-53de-483a-beb5-4a33c6065fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-d0dbe44a-53de-483a-beb5-4a33c6065fe0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:44.598873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:44.599804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:44.600703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 292/565] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:44.601987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbe44a-53de-483a-beb5-4a33c6065fe0 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:44.631441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeb8b6-d39a-406e-82ea-70eb92012102 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:44.631850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:44.667255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ef546-f103-4517-8e11-b97d2fa7fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:44.667780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ef546-f103-4517-8e11-b97d2fa7fc38 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.668102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ef546-f103-4517-8e11-b97d2fa7fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.668382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ef546-f103-4517-8e11-b97d2fa7fc38 None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.669068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ef546-f103-4517-8e11-b97d2fa7fc38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:44.669373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ef546-f103-4517-8e11-b97d2fa7fc38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:44.670808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ef546-f103-4517-8e11-b97d2fa7fc38 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:44.682911 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 274/566] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:44.683598 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ef546-f103-4517-8e11-b97d2fa7fc38 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:44.702760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbe44a-53de-483a-beb5-4a33c6065fe0 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:44.735462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c2a6-5491-44af-99c7-f44b862aed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:44.736109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c2a6-5491-44af-99c7-f44b862aed88 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.736395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c2a6-5491-44af-99c7-f44b862aed88 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.736676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c2a6-5491-44af-99c7-f44b862aed88 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.737331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659c2a6-5491-44af-99c7-f44b862aed88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:44.737632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659c2a6-5491-44af-99c7-f44b862aed88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:44.739005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659c2a6-5491-44af-99c7-f44b862aed88 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:44.739619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:44.750488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 293/567] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:44.751660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc256-9659-4d1e-8abb-6276b43ad6df None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.752049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc256-9659-4d1e-8abb-6276b43ad6df None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.752323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc256-9659-4d1e-8abb-6276b43ad6df None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.754047 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 294/568] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:44.754234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cc256-9659-4d1e-8abb-6276b43ad6df None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:44.788001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ef546-f103-4517-8e11-b97d2fa7fc38 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:44.822242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b64ef1-b083-433a-8961-28d8df1519bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:44.822869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b64ef1-b083-433a-8961-28d8df1519bc None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.823157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b64ef1-b083-433a-8961-28d8df1519bc None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.823438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b64ef1-b083-433a-8961-28d8df1519bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.833834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b64ef1-b083-433a-8961-28d8df1519bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:44.835273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b64ef1-b083-433a-8961-28d8df1519bc None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:44.845925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-d0b64ef1-b083-433a-8961-28d8df1519bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:44.849328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:44.855377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:44.856380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:44.856380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:44.856380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:44.856380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 275/569] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:44.856380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b64ef1-b083-433a-8961-28d8df1519bc None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:44.857549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cc256-9659-4d1e-8abb-6276b43ad6df None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:44.892406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56e39e-82e5-4c0b-aae4-b2d58c88d0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:44.893046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56e39e-82e5-4c0b-aae4-b2d58c88d0e7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.893285 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56e39e-82e5-4c0b-aae4-b2d58c88d0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.893562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56e39e-82e5-4c0b-aae4-b2d58c88d0e7 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.894252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56e39e-82e5-4c0b-aae4-b2d58c88d0e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:44.894569 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56e39e-82e5-4c0b-aae4-b2d58c88d0e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:44.895958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56e39e-82e5-4c0b-aae4-b2d58c88d0e7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:44.903712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 295/570] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:44.904531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56e39e-82e5-4c0b-aae4-b2d58c88d0e7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:44.955668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b64ef1-b083-433a-8961-28d8df1519bc None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:44.964668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:44.989056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b371f13-1b6d-474a-b041-8a40682e4b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:44.989664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b371f13-1b6d-474a-b041-8a40682e4b17 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:44.989956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b371f13-1b6d-474a-b041-8a40682e4b17 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:44.990236 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b371f13-1b6d-474a-b041-8a40682e4b17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:44.990963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b371f13-1b6d-474a-b041-8a40682e4b17 None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=1fae47d6b7a7472a9d94636bb126f802) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:44.991278 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b371f13-1b6d-474a-b041-8a40682e4b17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:44.993331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b371f13-1b6d-474a-b041-8a40682e4b17 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:44.999009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 276/571] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:44 2022] GET /identity/v3/role_assignments?user.id=1fae47d6b7a7472a9d94636bb126f802&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:45.000431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0bf2b-2a42-4582-8049-94a48c7e84fb None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:45.000712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0bf2b-2a42-4582-8049-94a48c7e84fb None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:45.000988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0bf2b-2a42-4582-8049-94a48c7e84fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:45.007266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56e39e-82e5-4c0b-aae4-b2d58c88d0e7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:45.020181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c7a0bf2b-2a42-4582-8049-94a48c7e84fb None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:45.021417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c7a0bf2b-2a42-4582-8049-94a48c7e84fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:45.048875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd57aac-15fc-467f-bc3b-3ddb7c629626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:45.049918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd57aac-15fc-467f-bc3b-3ddb7c629626 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:45.050420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd57aac-15fc-467f-bc3b-3ddb7c629626 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:45.050784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd57aac-15fc-467f-bc3b-3ddb7c629626 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:45.062493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd57aac-15fc-467f-bc3b-3ddb7c629626 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:45.064007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd57aac-15fc-467f-bc3b-3ddb7c629626 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-4bd57aac-15fc-467f-bc3b-3ddb7c629626 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:45.076157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:45.077252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 296/572] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:45.078311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd57aac-15fc-467f-bc3b-3ddb7c629626 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:45.096234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:45.115896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 277/573] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:45.115896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0bf2b-2a42-4582-8049-94a48c7e84fb None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:45.246264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd57aac-15fc-467f-bc3b-3ddb7c629626 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:45.246454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0bf2b-2a42-4582-8049-94a48c7e84fb None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:45.248148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:45.281478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b208a0-239a-4300-9f03-caac524a3c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:45.282224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b208a0-239a-4300-9f03-caac524a3c01 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:45.282572 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b208a0-239a-4300-9f03-caac524a3c01 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:45.282911 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b208a0-239a-4300-9f03-caac524a3c01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:45.290551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbdc38-32e4-4c40-98bc-688db6f45b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:45.291377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbdc38-32e4-4c40-98bc-688db6f45b1d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:45.292261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbdc38-32e4-4c40-98bc-688db6f45b1d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:45.293064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbdc38-32e4-4c40-98bc-688db6f45b1d None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:45.294690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbdc38-32e4-4c40-98bc-688db6f45b1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:45.295152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbdc38-32e4-4c40-98bc-688db6f45b1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:45.295355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b208a0-239a-4300-9f03-caac524a3c01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:45.296869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbdc38-32e4-4c40-98bc-688db6f45b1d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:45.296988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b208a0-239a-4300-9f03-caac524a3c01 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:45.312682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 297/574] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-e1b208a0-239a-4300-9f03-caac524a3c01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:45.312991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:45.314104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:45.315213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:45.315213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:45.315213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:45.316237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbdc38-32e4-4c40-98bc-688db6f45b1d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:45.323089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 278/575] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:45.323089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b208a0-239a-4300-9f03-caac524a3c01 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:45.422435 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:45.431723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b208a0-239a-4300-9f03-caac524a3c01 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:45.449568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbdc38-32e4-4c40-98bc-688db6f45b1d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:45.470240 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1460f-297f-41fe-8677-e68999c8a533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:45.470874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1460f-297f-41fe-8677-e68999c8a533 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:45.471160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1460f-297f-41fe-8677-e68999c8a533 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:45.471437 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1460f-297f-41fe-8677-e68999c8a533 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:45.480470 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1460f-297f-41fe-8677-e68999c8a533 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:45.481920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1460f-297f-41fe-8677-e68999c8a533 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:45.487846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc293a9-a6c8-4e33-8616-a90c5cb74ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:45.488578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc293a9-a6c8-4e33-8616-a90c5cb74ba7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:45.488975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc293a9-a6c8-4e33-8616-a90c5cb74ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:45.489346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc293a9-a6c8-4e33-8616-a90c5cb74ba7 None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:45.490173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc293a9-a6c8-4e33-8616-a90c5cb74ba7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:45.490580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc293a9-a6c8-4e33-8616-a90c5cb74ba7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:45.492149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc293a9-a6c8-4e33-8616-a90c5cb74ba7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-f3f1460f-297f-41fe-8677-e68999c8a533 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:45.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:45.494433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 279/576] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Aug 26 15:31:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:45.500578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1460f-297f-41fe-8677-e68999c8a533 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:45.515004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:45.518350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 298/577] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Aug 26 15:31:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:45.519349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc293a9-a6c8-4e33-8616-a90c5cb74ba7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:45.616385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1460f-297f-41fe-8677-e68999c8a533 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:45.626269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc293a9-a6c8-4e33-8616-a90c5cb74ba7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:45.653046 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cdeeb-a3d5-4498-8615-56251bf7769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:45.653911 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cdeeb-a3d5-4498-8615-56251bf7769a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:45.654296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cdeeb-a3d5-4498-8615-56251bf7769a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:45.654684 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cdeeb-a3d5-4498-8615-56251bf7769a None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:45.655404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196cdeeb-a3d5-4498-8615-56251bf7769a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:45.655789 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196cdeeb-a3d5-4498-8615-56251bf7769a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:45.657221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196cdeeb-a3d5-4498-8615-56251bf7769a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:45.658289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:45.663400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4783e-a4c1-4b79-a75c-c065942c9889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:45.664083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4783e-a4c1-4b79-a75c-c065942c9889 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:45.664371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4783e-a4c1-4b79-a75c-c065942c9889 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:45.664649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4783e-a4c1-4b79-a75c-c065942c9889 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:45.671366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 280/578] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:45.675227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4783e-a4c1-4b79-a75c-c065942c9889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:45.676486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cdeeb-a3d5-4498-8615-56251bf7769a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-b5c4783e-a4c1-4b79-a75c-c065942c9889 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:45.680610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:45.681750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:45.682794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:45.686516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 299/579] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:45.687332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4783e-a4c1-4b79-a75c-c065942c9889 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:45.755385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:45.780923 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cdeeb-a3d5-4498-8615-56251bf7769a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:45.798240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4783e-a4c1-4b79-a75c-c065942c9889 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:45.815957 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97241b98-94b7-4efb-812e-d26780ebc8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:45.816573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97241b98-94b7-4efb-812e-d26780ebc8ee None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:45.816867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97241b98-94b7-4efb-812e-d26780ebc8ee None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:45.817149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97241b98-94b7-4efb-812e-d26780ebc8ee None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:45.817885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97241b98-94b7-4efb-812e-d26780ebc8ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:45.818200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97241b98-94b7-4efb-812e-d26780ebc8ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:45.819639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97241b98-94b7-4efb-812e-d26780ebc8ee None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:45.834974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b53856-11fb-4c2a-8836-ecc49e62af32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:45.836430 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b53856-11fb-4c2a-8836-ecc49e62af32 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:45.836719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b53856-11fb-4c2a-8836-ecc49e62af32 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:45.836992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b53856-11fb-4c2a-8836-ecc49e62af32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:45.837875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 281/580] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Aug 26 15:31:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:45.838469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97241b98-94b7-4efb-812e-d26780ebc8ee None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:45.851640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b53856-11fb-4c2a-8836-ecc49e62af32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:45.853135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b53856-11fb-4c2a-8836-ecc49e62af32 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-b6b53856-11fb-4c2a-8836-ecc49e62af32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:45.867704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:45.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:45.869854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:45.869854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:45.869854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:45.869854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 300/581] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:45.869854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b53856-11fb-4c2a-8836-ecc49e62af32 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:45.914756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:45.952120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97241b98-94b7-4efb-812e-d26780ebc8ee None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:45.971523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b53856-11fb-4c2a-8836-ecc49e62af32 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:45.984380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:45.991588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad7e0-548e-442e-a482-7461a5a46923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:45.992978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad7e0-548e-442e-a482-7461a5a46923 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:45.993761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad7e0-548e-442e-a482-7461a5a46923 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:45.994098 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad7e0-548e-442e-a482-7461a5a46923 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.001731 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ad7e0-548e-442e-a482-7461a5a46923 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.003262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ad7e0-548e-442e-a482-7461a5a46923 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.008562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434002bd-ea05-457a-bfc1-99f2b5c9bdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:46.009114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434002bd-ea05-457a-bfc1-99f2b5c9bdb7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:46.009393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434002bd-ea05-457a-bfc1-99f2b5c9bdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:46.009667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434002bd-ea05-457a-bfc1-99f2b5c9bdb7 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.010452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434002bd-ea05-457a-bfc1-99f2b5c9bdb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:46.010762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434002bd-ea05-457a-bfc1-99f2b5c9bdb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.012153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434002bd-ea05-457a-bfc1-99f2b5c9bdb7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-e21ad7e0-548e-442e-a482-7461a5a46923 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:46.016758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:46.017865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:46.018953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:46.018953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:46.018953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:46.018953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:46.018953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:46.018953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:46.018953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:46.018953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:46.018953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:31:46.018953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:46.018953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 282/582] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:46.019557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad7e0-548e-442e-a482-7461a5a46923 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.030546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 301/583] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:46.030546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434002bd-ea05-457a-bfc1-99f2b5c9bdb7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.138740 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad7e0-548e-442e-a482-7461a5a46923 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:46.146186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434002bd-ea05-457a-bfc1-99f2b5c9bdb7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:46.154895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:46.176211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe98aa-ca53-4725-9e98-6d599dd40724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:46.176924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe98aa-ca53-4725-9e98-6d599dd40724 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:46.177505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe98aa-ca53-4725-9e98-6d599dd40724 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:46.177619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe98aa-ca53-4725-9e98-6d599dd40724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.183368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902d8ae-2345-45d0-834c-58de9c0f79ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:46.184165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902d8ae-2345-45d0-834c-58de9c0f79ae None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:46.184521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902d8ae-2345-45d0-834c-58de9c0f79ae None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:46.184861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902d8ae-2345-45d0-834c-58de9c0f79ae None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.185574 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902d8ae-2345-45d0-834c-58de9c0f79ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:46.185948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902d8ae-2345-45d0-834c-58de9c0f79ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.188226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902d8ae-2345-45d0-834c-58de9c0f79ae None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.190087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe98aa-ca53-4725-9e98-6d599dd40724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.191690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe98aa-ca53-4725-9e98-6d599dd40724 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.201667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 302/584] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:46.202642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902d8ae-2345-45d0-834c-58de9c0f79ae None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-5afe98aa-ca53-4725-9e98-6d599dd40724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:46.211159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:46.212317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:46.217838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:46.217838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:46.217838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:46.217838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 283/585] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:46.217838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe98aa-ca53-4725-9e98-6d599dd40724 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.267820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:46.313709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902d8ae-2345-45d0-834c-58de9c0f79ae None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:46.327477 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe98aa-ca53-4725-9e98-6d599dd40724 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:46.348664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a385d-ded3-462d-b162-e5ea4276be04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:46.349428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a385d-ded3-462d-b162-e5ea4276be04 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:46.349683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a385d-ded3-462d-b162-e5ea4276be04 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:46.350025 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a385d-ded3-462d-b162-e5ea4276be04 None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.350865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a385d-ded3-462d-b162-e5ea4276be04 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:46.351281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a385d-ded3-462d-b162-e5ea4276be04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.352862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a385d-ded3-462d-b162-e5ea4276be04 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.362057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68884192-64ea-4231-a2ea-4569350b9c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:46.362953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68884192-64ea-4231-a2ea-4569350b9c8e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:46.362953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68884192-64ea-4231-a2ea-4569350b9c8e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:46.363206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68884192-64ea-4231-a2ea-4569350b9c8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.371346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 303/586] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:46.372195 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a385d-ded3-462d-b162-e5ea4276be04 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.376816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68884192-64ea-4231-a2ea-4569350b9c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.378284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68884192-64ea-4231-a2ea-4569350b9c8e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-68884192-64ea-4231-a2ea-4569350b9c8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:46.394184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:46.395220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:46.396229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:46.396229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:46.396229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:46.396229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 284/587] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:46.396229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68884192-64ea-4231-a2ea-4569350b9c8e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.402888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:46.483963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a385d-ded3-462d-b162-e5ea4276be04 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:46.504301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:46.511453 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68884192-64ea-4231-a2ea-4569350b9c8e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:46.521307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42ae68-0f53-4a72-8f2a-fcf09d8e6825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:46.521307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42ae68-0f53-4a72-8f2a-fcf09d8e6825 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:46.521521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42ae68-0f53-4a72-8f2a-fcf09d8e6825 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:46.521714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42ae68-0f53-4a72-8f2a-fcf09d8e6825 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.528824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42ae68-0f53-4a72-8f2a-fcf09d8e6825 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.530271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42ae68-0f53-4a72-8f2a-fcf09d8e6825 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-1d42ae68-0f53-4a72-8f2a-fcf09d8e6825 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:46.541820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:46.542737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:46.543683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:46.543683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:46.543683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:46.543683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:46.543683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:46.543683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:46.543683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:46.543683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:46.543683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 304/588] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:46.548403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42ae68-0f53-4a72-8f2a-fcf09d8e6825 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.550540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0aa02-7f13-4cfd-9ed9-cc87967ccb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:46.551166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0aa02-7f13-4cfd-9ed9-cc87967ccb9a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:46.551451 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0aa02-7f13-4cfd-9ed9-cc87967ccb9a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:46.552517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0aa02-7f13-4cfd-9ed9-cc87967ccb9a None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.552517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0aa02-7f13-4cfd-9ed9-cc87967ccb9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:46.552798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0aa02-7f13-4cfd-9ed9-cc87967ccb9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.554972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0aa02-7f13-4cfd-9ed9-cc87967ccb9a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.571541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 285/589] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:46.571541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0aa02-7f13-4cfd-9ed9-cc87967ccb9a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.650808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:46.669286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42ae68-0f53-4a72-8f2a-fcf09d8e6825 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:46.683743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0aa02-7f13-4cfd-9ed9-cc87967ccb9a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:46.706223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4dd2d-c1eb-47d3-90e8-1d8ff15bf512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:46.706939 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4dd2d-c1eb-47d3-90e8-1d8ff15bf512 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:46.707307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4dd2d-c1eb-47d3-90e8-1d8ff15bf512 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:46.707656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4dd2d-c1eb-47d3-90e8-1d8ff15bf512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.717768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4dd2d-c1eb-47d3-90e8-1d8ff15bf512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.718638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b86939-0899-4b40-9e24-9b5d90945d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:46.719316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b86939-0899-4b40-9e24-9b5d90945d18 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:46.719413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4dd2d-c1eb-47d3-90e8-1d8ff15bf512 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.719732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b86939-0899-4b40-9e24-9b5d90945d18 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:46.720120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b86939-0899-4b40-9e24-9b5d90945d18 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.720819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b86939-0899-4b40-9e24-9b5d90945d18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:46.721192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b86939-0899-4b40-9e24-9b5d90945d18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.722539 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b86939-0899-4b40-9e24-9b5d90945d18 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.738876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 286/590] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-69e4dd2d-c1eb-47d3-90e8-1d8ff15bf512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:46.739036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:46.740340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:46.741381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:46.741381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:46.741381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:46.742150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b86939-0899-4b40-9e24-9b5d90945d18 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.742255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 305/591] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:46.742255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4dd2d-c1eb-47d3-90e8-1d8ff15bf512 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.759564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:46.843119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4dd2d-c1eb-47d3-90e8-1d8ff15bf512 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:46.844590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b86939-0899-4b40-9e24-9b5d90945d18 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:46.881838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67c28e-a07c-45a3-8c5b-7c7db9bbb4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:46.882302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9875f-94fd-48de-a17f-fa79c2a9f82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:46.882515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67c28e-a07c-45a3-8c5b-7c7db9bbb4b5 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:46.882775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67c28e-a07c-45a3-8c5b-7c7db9bbb4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:46.882935 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9875f-94fd-48de-a17f-fa79c2a9f82b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:46.883041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67c28e-a07c-45a3-8c5b-7c7db9bbb4b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.883190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9875f-94fd-48de-a17f-fa79c2a9f82b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:46.883456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9875f-94fd-48de-a17f-fa79c2a9f82b None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:46.884174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9875f-94fd-48de-a17f-fa79c2a9f82b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:46.884485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9875f-94fd-48de-a17f-fa79c2a9f82b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.885953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9875f-94fd-48de-a17f-fa79c2a9f82b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.886234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:46.890758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67c28e-a07c-45a3-8c5b-7c7db9bbb4b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:46.893803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67c28e-a07c-45a3-8c5b-7c7db9bbb4b5 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:46.898385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 287/592] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:46.899319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9875f-94fd-48de-a17f-fa79c2a9f82b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-fc67c28e-a07c-45a3-8c5b-7c7db9bbb4b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:46.908746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:46.909813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:46.910910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:46.910910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:46.910910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:46.910910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:46.910910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:46.910910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:46.910910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:46.910910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:46.912741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 306/593] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:46.913706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67c28e-a07c-45a3-8c5b-7c7db9bbb4b5 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:46.989807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:47.013169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9875f-94fd-48de-a17f-fa79c2a9f82b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:47.019788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67c28e-a07c-45a3-8c5b-7c7db9bbb4b5 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:47.048461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fee2f-f914-4ca8-87cf-0e44c63ab1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:47.049185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fee2f-f914-4ca8-87cf-0e44c63ab1f2 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:47.049518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fee2f-f914-4ca8-87cf-0e44c63ab1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:47.049842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fee2f-f914-4ca8-87cf-0e44c63ab1f2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:47.055292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6989aa-6960-4b2b-b59c-1b1667b5a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:47.055918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6989aa-6960-4b2b-b59c-1b1667b5a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:47.056219 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6989aa-6960-4b2b-b59c-1b1667b5a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:47.056479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6989aa-6960-4b2b-b59c-1b1667b5a8c7 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:47.057118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6989aa-6960-4b2b-b59c-1b1667b5a8c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:47.057424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6989aa-6960-4b2b-b59c-1b1667b5a8c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:47.058239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fee2f-f914-4ca8-87cf-0e44c63ab1f2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:47.058789 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6989aa-6960-4b2b-b59c-1b1667b5a8c7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:47.059663 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fee2f-f914-4ca8-87cf-0e44c63ab1f2 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-0a0fee2f-f914-4ca8-87cf-0e44c63ab1f2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:47.071180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:47.072197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:47.073192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:47.073192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:47.073192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:47.073192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:47.073192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:47.073192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:47.073192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:47.073192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:47.073192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:31:47.073192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:47.073732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 307/594] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:47.073732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6989aa-6960-4b2b-b59c-1b1667b5a8c7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:47.073863 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 288/595] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:47.073863 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fee2f-f914-4ca8-87cf-0e44c63ab1f2 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:47.129027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:47.172697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6989aa-6960-4b2b-b59c-1b1667b5a8c7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:47.175830 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fee2f-f914-4ca8-87cf-0e44c63ab1f2 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:47.206202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3428857-f494-4a9d-92f3-32e8ac4ce8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:47.206852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3428857-f494-4a9d-92f3-32e8ac4ce8e7 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:47.207135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3428857-f494-4a9d-92f3-32e8ac4ce8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:47.207412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3428857-f494-4a9d-92f3-32e8ac4ce8e7 None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/cef193a54eff44f2b53716e3ec1a29cf/roles/9dc2d706af5146e0b8d62ef94c74906d` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:47.208550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061edc6-f915-40c0-b30b-7ff42ad01afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:47.209151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061edc6-f915-40c0-b30b-7ff42ad01afc None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:47.209435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061edc6-f915-40c0-b30b-7ff42ad01afc None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:47.209715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061edc6-f915-40c0-b30b-7ff42ad01afc None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:47.210429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061edc6-f915-40c0-b30b-7ff42ad01afc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:47.210738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061edc6-f915-40c0-b30b-7ff42ad01afc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:47.212185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061edc6-f915-40c0-b30b-7ff42ad01afc None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:47.214602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:47.221516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3428857-f494-4a9d-92f3-32e8ac4ce8e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=cef193a54eff44f2b53716e3ec1a29cf, role_id=9dc2d706af5146e0b8d62ef94c74906d)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:47.222948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3428857-f494-4a9d-92f3-32e8ac4ce8e7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:47.229594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 289/596] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:47.229594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddcb56-022f-4938-bd6d-a1e37d8b54c3 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:47.229594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddcb56-022f-4938-bd6d-a1e37d8b54c3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:47.229594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddcb56-022f-4938-bd6d-a1e37d8b54c3 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:47.229594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 290/597] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:47.229594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddcb56-022f-4938-bd6d-a1e37d8b54c3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:47.281507 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 308/598] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:31:47 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/cef193a54eff44f2b53716e3ec1a29cf/roles/9dc2d706af5146e0b8d62ef94c74906d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:31:47.282334 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3428857-f494-4a9d-92f3-32e8ac4ce8e7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:47.341892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddcb56-022f-4938-bd6d-a1e37d8b54c3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:47.380794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e3f62-9ba0-4c5c-940a-0b88edabfc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:47.381693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e3f62-9ba0-4c5c-940a-0b88edabfc53 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:47.382294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e3f62-9ba0-4c5c-940a-0b88edabfc53 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:47.382988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e3f62-9ba0-4c5c-940a-0b88edabfc53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:47.390255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e3f62-9ba0-4c5c-940a-0b88edabfc53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:47.391744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e3f62-9ba0-4c5c-940a-0b88edabfc53 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-bb6e3f62-9ba0-4c5c-940a-0b88edabfc53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:47.403800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:47.404833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:47.405741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:47.405741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:47.405741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:47.405741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:47.405741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:47.405741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:47.405741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:47.405741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:47.405741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 291/599] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:47.405741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e3f62-9ba0-4c5c-940a-0b88edabfc53 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:47.411632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3428857-f494-4a9d-92f3-32e8ac4ce8e7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:47.422813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:47.454883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4b6f1-1264-469b-9a82-74b070f30de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:47.455502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4b6f1-1264-469b-9a82-74b070f30de9 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:47.455785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4b6f1-1264-469b-9a82-74b070f30de9 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:47.456093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4b6f1-1264-469b-9a82-74b070f30de9 None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:47.457181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4b6f1-1264-469b-9a82-74b070f30de9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:47.457492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4b6f1-1264-469b-9a82-74b070f30de9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:47.459248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4b6f1-1264-469b-9a82-74b070f30de9 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:47.472652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:47.479737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 309/600] 10.222.0.98 () {56 vars in 1139 bytes} [Fri Aug 26 15:31:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:47.480953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea394bd-b101-445e-9412-2d4b8d8d9334 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:47.481243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea394bd-b101-445e-9412-2d4b8d8d9334 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:47.481521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea394bd-b101-445e-9412-2d4b8d8d9334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:47.499512 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-fea394bd-b101-445e-9412-2d4b8d8d9334 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:47.500763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-fea394bd-b101-445e-9412-2d4b8d8d9334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:47.521438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e3f62-9ba0-4c5c-940a-0b88edabfc53 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:47.561834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfac863-ccfb-4046-beef-cfd5f00e60fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:47.562497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfac863-ccfb-4046-beef-cfd5f00e60fc None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:47.562781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfac863-ccfb-4046-beef-cfd5f00e60fc None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:47.563055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfac863-ccfb-4046-beef-cfd5f00e60fc None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/d1b01c21c38949938141eefdc4939385/roles/9dc2d706af5146e0b8d62ef94c74906d` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:47.579891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 310/601] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:47.579891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea394bd-b101-445e-9412-2d4b8d8d9334 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:47.586134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfac863-ccfb-4046-beef-cfd5f00e60fc None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=d1b01c21c38949938141eefdc4939385, role_id=9dc2d706af5146e0b8d62ef94c74906d)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:47.587729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfac863-ccfb-4046-beef-cfd5f00e60fc None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:47.676235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 292/602] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:31:47 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/d1b01c21c38949938141eefdc4939385/roles/9dc2d706af5146e0b8d62ef94c74906d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:31:47.677106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfac863-ccfb-4046-beef-cfd5f00e60fc None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:47.690047 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:47.703028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea394bd-b101-445e-9412-2d4b8d8d9334 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:47.790483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e246d0-14c7-4fbc-be97-4e3729cd73a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:47.791104 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e246d0-14c7-4fbc-be97-4e3729cd73a7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:47.791387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e246d0-14c7-4fbc-be97-4e3729cd73a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:47.791662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e246d0-14c7-4fbc-be97-4e3729cd73a7 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:47.797352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e246d0-14c7-4fbc-be97-4e3729cd73a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:47.797352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e246d0-14c7-4fbc-be97-4e3729cd73a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:47.797352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e246d0-14c7-4fbc-be97-4e3729cd73a7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:47.820879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 311/603] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:47.826644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e246d0-14c7-4fbc-be97-4e3729cd73a7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:47.837680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfac863-ccfb-4046-beef-cfd5f00e60fc None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:47.849488 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:47.882602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ef0d8-4555-4761-bd37-07eaed3b0712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:47.883542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ef0d8-4555-4761-bd37-07eaed3b0712 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:47.884000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ef0d8-4555-4761-bd37-07eaed3b0712 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:47.884436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ef0d8-4555-4761-bd37-07eaed3b0712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:47.901597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ef0d8-4555-4761-bd37-07eaed3b0712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:47.903718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ef0d8-4555-4761-bd37-07eaed3b0712 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-320ef0d8-4555-4761-bd37-07eaed3b0712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:47.920727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:47.921737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:47.922641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:47.922641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:47.922641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:47.922641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 293/604] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:47.922641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ef0d8-4555-4761-bd37-07eaed3b0712 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:47.985066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e246d0-14c7-4fbc-be97-4e3729cd73a7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:48.030480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ebde2-2e90-4950-9d50-63825d722d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:48.031703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ebde2-2e90-4950-9d50-63825d722d48 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:48.032030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ebde2-2e90-4950-9d50-63825d722d48 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:48.032310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ebde2-2e90-4950-9d50-63825d722d48 None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/3a1ff8aff9e048c6972843e0b50c4d45/roles/9dc2d706af5146e0b8d62ef94c74906d` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:48.033325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ef0d8-4555-4761-bd37-07eaed3b0712 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:48.047543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ebde2-2e90-4950-9d50-63825d722d48 None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=3a1ff8aff9e048c6972843e0b50c4d45, role_id=9dc2d706af5146e0b8d62ef94c74906d)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:48.049100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ebde2-2e90-4950-9d50-63825d722d48 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:48.069021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b28903-bca5-466f-9759-15a941e5a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:48.070185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b28903-bca5-466f-9759-15a941e5a16c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:48.070466 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b28903-bca5-466f-9759-15a941e5a16c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:48.070765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b28903-bca5-466f-9759-15a941e5a16c None admin] PATH_INFO: `/v3/roles/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:48.081105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b28903-bca5-466f-9759-15a941e5a16c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-86b28903-bca5-466f-9759-15a941e5a16c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:48.086767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:48.087721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:48.088638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 294/605] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:48.089417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b28903-bca5-466f-9759-15a941e5a16c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:48.111694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:48.147454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 312/606] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:31:47 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/3a1ff8aff9e048c6972843e0b50c4d45/roles/9dc2d706af5146e0b8d62ef94c74906d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:31:48.152546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ebde2-2e90-4950-9d50-63825d722d48 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:48.159144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:48.257827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b28903-bca5-466f-9759-15a941e5a16c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:48.308358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ebde2-2e90-4950-9d50-63825d722d48 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:48.318972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19272a61-3f58-4389-b814-649450e6af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:48.320306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19272a61-3f58-4389-b814-649450e6af74 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:48.320656 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19272a61-3f58-4389-b814-649450e6af74 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:48.321639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19272a61-3f58-4389-b814-649450e6af74 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:48.322376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19272a61-3f58-4389-b814-649450e6af74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:48.322983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19272a61-3f58-4389-b814-649450e6af74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:48.324474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19272a61-3f58-4389-b814-649450e6af74 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:48.347153 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 295/607] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:48.352937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19272a61-3f58-4389-b814-649450e6af74 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:48.371385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a37af9-bedc-400f-b22e-54b3ca8a60a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:48.372905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a37af9-bedc-400f-b22e-54b3ca8a60a5 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:48.373678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a37af9-bedc-400f-b22e-54b3ca8a60a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:48.374168 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a37af9-bedc-400f-b22e-54b3ca8a60a5 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:48.375291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a37af9-bedc-400f-b22e-54b3ca8a60a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:48.376196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a37af9-bedc-400f-b22e-54b3ca8a60a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:48.377838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a37af9-bedc-400f-b22e-54b3ca8a60a5 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:48.391244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 313/608] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:48.397871 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a37af9-bedc-400f-b22e-54b3ca8a60a5 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:48.446950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:48.480808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19272a61-3f58-4389-b814-649450e6af74 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:48.493563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:48.546051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65668d-df87-45ba-ba36-f8f63d2014ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:48.547022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65668d-df87-45ba-ba36-f8f63d2014ff None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:48.548197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65668d-df87-45ba-ba36-f8f63d2014ff None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:48.549283 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65668d-df87-45ba-ba36-f8f63d2014ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:48.561347 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65668d-df87-45ba-ba36-f8f63d2014ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:48.564375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65668d-df87-45ba-ba36-f8f63d2014ff None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:48.566734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a37af9-bedc-400f-b22e-54b3ca8a60a5 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-ce65668d-df87-45ba-ba36-f8f63d2014ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:48.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:48.594765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:48.595744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:48.595744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:48.595744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:48.595744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:48.595744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:48.595744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:48.595744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:48.595744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:48.595744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 296/609] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:48.596787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65668d-df87-45ba-ba36-f8f63d2014ff None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:48.638106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28626ef0-a1d2-402c-a96f-40ad54f68fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:48.638748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28626ef0-a1d2-402c-a96f-40ad54f68fa3 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:48.639029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28626ef0-a1d2-402c-a96f-40ad54f68fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:48.639302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28626ef0-a1d2-402c-a96f-40ad54f68fa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:48.657002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28626ef0-a1d2-402c-a96f-40ad54f68fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:48.658441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28626ef0-a1d2-402c-a96f-40ad54f68fa3 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-28626ef0-a1d2-402c-a96f-40ad54f68fa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:48.680680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:48.681692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:48.682600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:48.682600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:48.682600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:48.682600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 314/610] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:48.688159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28626ef0-a1d2-402c-a96f-40ad54f68fa3 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:48.809034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65668d-df87-45ba-ba36-f8f63d2014ff None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:48.858890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28626ef0-a1d2-402c-a96f-40ad54f68fa3 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:48.879160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:48.884579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccb4a5-1a32-47bd-9840-0fce10d33461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:48.885200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb4a5-1a32-47bd-9840-0fce10d33461 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:48.885482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb4a5-1a32-47bd-9840-0fce10d33461 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:48.885761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb4a5-1a32-47bd-9840-0fce10d33461 None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:48.886462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccb4a5-1a32-47bd-9840-0fce10d33461 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:48.886773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccb4a5-1a32-47bd-9840-0fce10d33461 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:48.888247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccb4a5-1a32-47bd-9840-0fce10d33461 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:48.897726 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4cd6e-fdf2-49cd-ab8e-eeb5d6f2f178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:48.898700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:48.898812 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4cd6e-fdf2-49cd-ab8e-eeb5d6f2f178 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:48.899119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4cd6e-fdf2-49cd-ab8e-eeb5d6f2f178 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:48.899339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4cd6e-fdf2-49cd-ab8e-eeb5d6f2f178 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:48.900309 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4cd6e-fdf2-49cd-ab8e-eeb5d6f2f178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:48.900629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4cd6e-fdf2-49cd-ab8e-eeb5d6f2f178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:48.902077 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4cd6e-fdf2-49cd-ab8e-eeb5d6f2f178 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:48.908753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 297/611] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:48.914383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccb4a5-1a32-47bd-9840-0fce10d33461 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:48.918132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 315/612] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:48.926964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4cd6e-fdf2-49cd-ab8e-eeb5d6f2f178 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:49.059446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4cd6e-fdf2-49cd-ab8e-eeb5d6f2f178 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:49.098043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47b86d-cc6a-4ea6-91c6-f7d5f9a52d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:49.098686 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47b86d-cc6a-4ea6-91c6-f7d5f9a52d40 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:49.098974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47b86d-cc6a-4ea6-91c6-f7d5f9a52d40 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:49.099425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47b86d-cc6a-4ea6-91c6-f7d5f9a52d40 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:49.108536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47b86d-cc6a-4ea6-91c6-f7d5f9a52d40 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:49.110284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47b86d-cc6a-4ea6-91c6-f7d5f9a52d40 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-6b47b86d-cc6a-4ea6-91c6-f7d5f9a52d40 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:49.123925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:49.126087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:49.128340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:49.128340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:49.128340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:49.128340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:49.128340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:49.128340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:49.128340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:49.128340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:49.128340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:31:49.128340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:49.128340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 316/613] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:31:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:49.135913 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47b86d-cc6a-4ea6-91c6-f7d5f9a52d40 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:49.173312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccb4a5-1a32-47bd-9840-0fce10d33461 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:49.177672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:49.263293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47b86d-cc6a-4ea6-91c6-f7d5f9a52d40 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:49.267979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05529da-3871-47e8-a8b0-9d0c79bd71a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:49.268665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05529da-3871-47e8-a8b0-9d0c79bd71a7 None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:49.268944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05529da-3871-47e8-a8b0-9d0c79bd71a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:49.269218 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05529da-3871-47e8-a8b0-9d0c79bd71a7 None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/8c6cad18359143868276eb9d79a3fd3b/roles/9dc2d706af5146e0b8d62ef94c74906d` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:49.301798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8f6d3-0ebe-4d88-868f-f3535e991069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:49.302750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f6d3-0ebe-4d88-868f-f3535e991069 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:49.303302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f6d3-0ebe-4d88-868f-f3535e991069 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:49.303621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f6d3-0ebe-4d88-868f-f3535e991069 None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:49.304402 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8f6d3-0ebe-4d88-868f-f3535e991069 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:49.304720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8f6d3-0ebe-4d88-868f-f3535e991069 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:49.306336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8f6d3-0ebe-4d88-868f-f3535e991069 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:49.312092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05529da-3871-47e8-a8b0-9d0c79bd71a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=8c6cad18359143868276eb9d79a3fd3b, role_id=9dc2d706af5146e0b8d62ef94c74906d)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:49.313481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05529da-3871-47e8-a8b0-9d0c79bd71a7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:49.332448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 317/614] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Aug 26 15:31:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:49.338611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8f6d3-0ebe-4d88-868f-f3535e991069 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:49.388690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 298/615] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:31:48 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/8c6cad18359143868276eb9d79a3fd3b/roles/9dc2d706af5146e0b8d62ef94c74906d => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:31:49.392075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6d60-9220-47df-a39c-7663f11c603a None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:49.396141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6d60-9220-47df-a39c-7663f11c603a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:49.396141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6d60-9220-47df-a39c-7663f11c603a None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:49.396141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 299/616] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:49.404481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113ef24-64e7-4998-978e-630751794ad3 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:49.404809 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113ef24-64e7-4998-978e-630751794ad3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:49.405443 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113ef24-64e7-4998-978e-630751794ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:49.427269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-a113ef24-64e7-4998-978e-630751794ad3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:49.429291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a113ef24-64e7-4998-978e-630751794ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:49.509993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:49.513410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:49.517047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 300/617] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:49.534027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8f6d3-0ebe-4d88-868f-f3535e991069 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:49.574617 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f8857-7218-4f5a-8094-bed7084f6ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:49.578183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f8857-7218-4f5a-8094-bed7084f6ee0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:49.578183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f8857-7218-4f5a-8094-bed7084f6ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:49.578183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f8857-7218-4f5a-8094-bed7084f6ee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:49.589149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f8857-7218-4f5a-8094-bed7084f6ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:49.590603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f8857-7218-4f5a-8094-bed7084f6ee0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-a46f8857-7218-4f5a-8094-bed7084f6ee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:49.607657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:49.608979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:49.609999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:49.609999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:49.609999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:49.611952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 318/618] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:49.617812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113ef24-64e7-4998-978e-630751794ad3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:49.734876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113ef24-64e7-4998-978e-630751794ad3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:49.769456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca720a-32f0-410c-8cb2-0697ea062bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:49.770069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca720a-32f0-410c-8cb2-0697ea062bf3 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:49.770358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca720a-32f0-410c-8cb2-0697ea062bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:49.770624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca720a-32f0-410c-8cb2-0697ea062bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:49.771271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca720a-32f0-410c-8cb2-0697ea062bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:49.771578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca720a-32f0-410c-8cb2-0697ea062bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:49.773517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca720a-32f0-410c-8cb2-0697ea062bf3 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:49.785175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 301/619] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:49.790864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f8857-7218-4f5a-8094-bed7084f6ee0 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:49.858302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca720a-32f0-410c-8cb2-0697ea062bf3 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:49.929905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f8857-7218-4f5a-8094-bed7084f6ee0 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:49.940220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:49.969506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13b370-a207-4a41-8e82-cac67e8000f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:49.970143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13b370-a207-4a41-8e82-cac67e8000f0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:49.970453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13b370-a207-4a41-8e82-cac67e8000f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:49.970712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13b370-a207-4a41-8e82-cac67e8000f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:49.983109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13b370-a207-4a41-8e82-cac67e8000f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:49.984599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13b370-a207-4a41-8e82-cac67e8000f0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-8d13b370-a207-4a41-8e82-cac67e8000f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:49.998332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:49.999270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:50.000610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:50.000610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:50.000610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:50.000610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:50.000610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:50.000610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:50.000610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:50.000610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:50.000610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 319/620] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:50.005358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13b370-a207-4a41-8e82-cac67e8000f0 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:50.032289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:50.061918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca720a-32f0-410c-8cb2-0697ea062bf3 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:50.100516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8048d0-7104-46ab-a02c-f6fbd7422b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:50.101319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8048d0-7104-46ab-a02c-f6fbd7422b9c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:50.101722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8048d0-7104-46ab-a02c-f6fbd7422b9c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:50.102086 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8048d0-7104-46ab-a02c-f6fbd7422b9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:50.113283 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8048d0-7104-46ab-a02c-f6fbd7422b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:50.115141 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13b370-a207-4a41-8e82-cac67e8000f0 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-ee8048d0-7104-46ab-a02c-f6fbd7422b9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:50.120226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:50.121369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:50.122231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:50.123120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 302/621] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:50.126544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff00a77-dac9-4de5-8e54-e69b2ac3ab69 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:50.126544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff00a77-dac9-4de5-8e54-e69b2ac3ab69 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:50.126544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff00a77-dac9-4de5-8e54-e69b2ac3ab69 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:50.126870 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 303/622] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:50.129511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff00a77-dac9-4de5-8e54-e69b2ac3ab69 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:50.157024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac2da0-8c50-41d9-9389-8b86c8eed887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:50.157829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac2da0-8c50-41d9-9389-8b86c8eed887 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:50.158212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac2da0-8c50-41d9-9389-8b86c8eed887 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:50.158627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac2da0-8c50-41d9-9389-8b86c8eed887 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:50.159428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac2da0-8c50-41d9-9389-8b86c8eed887 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:50.159904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac2da0-8c50-41d9-9389-8b86c8eed887 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:50.161446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac2da0-8c50-41d9-9389-8b86c8eed887 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:50.179797 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 320/623] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:50.181357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428071c-8c4c-49cd-9c47-02480df677a3 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:50.181750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428071c-8c4c-49cd-9c47-02480df677a3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:50.182138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428071c-8c4c-49cd-9c47-02480df677a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:50.201397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1428071c-8c4c-49cd-9c47-02480df677a3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:50.202996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1428071c-8c4c-49cd-9c47-02480df677a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:50.254719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:50.255937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff00a77-dac9-4de5-8e54-e69b2ac3ab69 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:50.271856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:50.279965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 321/624] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:50.279965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428071c-8c4c-49cd-9c47-02480df677a3 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:50.320324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c312e7-e708-49e0-9b95-26339c871009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:50.320985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c312e7-e708-49e0-9b95-26339c871009 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:50.321330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c312e7-e708-49e0-9b95-26339c871009 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:50.321618 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c312e7-e708-49e0-9b95-26339c871009 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:50.322296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c312e7-e708-49e0-9b95-26339c871009 None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:50.322601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c312e7-e708-49e0-9b95-26339c871009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:50.325359 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c312e7-e708-49e0-9b95-26339c871009 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:50.344180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 304/625] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:50.349051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c312e7-e708-49e0-9b95-26339c871009 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:50.445658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428071c-8c4c-49cd-9c47-02480df677a3 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:50.485244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44433758-c54f-43a9-b7ef-62eed75bf130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:50.485914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433758-c54f-43a9-b7ef-62eed75bf130 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:50.486198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433758-c54f-43a9-b7ef-62eed75bf130 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:50.486476 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433758-c54f-43a9-b7ef-62eed75bf130 None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/1fae47d6b7a7472a9d94636bb126f802/roles/9dc2d706af5146e0b8d62ef94c74906d` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:50.523284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44433758-c54f-43a9-b7ef-62eed75bf130 None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=1fae47d6b7a7472a9d94636bb126f802, role_id=9dc2d706af5146e0b8d62ef94c74906d)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:50.524827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44433758-c54f-43a9-b7ef-62eed75bf130 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:50.548652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c312e7-e708-49e0-9b95-26339c871009 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:50.589128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8db46-c79d-42e5-8739-ea8453a0e761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:50.589767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8db46-c79d-42e5-8739-ea8453a0e761 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:50.590048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8db46-c79d-42e5-8739-ea8453a0e761 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:50.590326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8db46-c79d-42e5-8739-ea8453a0e761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:50.604715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8db46-c79d-42e5-8739-ea8453a0e761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:50.606228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8db46-c79d-42e5-8739-ea8453a0e761 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-57a8db46-c79d-42e5-8739-ea8453a0e761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:50.623159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:50.624414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:50.625790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:50.625790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:50.625790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:50.625790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 305/626] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:50.626034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 322/627] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:31:50 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/1fae47d6b7a7472a9d94636bb126f802/roles/9dc2d706af5146e0b8d62ef94c74906d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:31:50.626034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44433758-c54f-43a9-b7ef-62eed75bf130 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:50.629097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e89d75-ea57-4a02-ba51-7830e1360ee1 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:50.629391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e89d75-ea57-4a02-ba51-7830e1360ee1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:50.629676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e89d75-ea57-4a02-ba51-7830e1360ee1 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:50.634718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 306/628] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:50.634718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e89d75-ea57-4a02-ba51-7830e1360ee1 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:50.654978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:50.691287 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:50.752676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44433758-c54f-43a9-b7ef-62eed75bf130 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:50.763109 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e89d75-ea57-4a02-ba51-7830e1360ee1 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:50.798255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabbe54-a00f-4979-8379-435a41188231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:50.798943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabbe54-a00f-4979-8379-435a41188231 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:50.799224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabbe54-a00f-4979-8379-435a41188231 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:50.799499 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabbe54-a00f-4979-8379-435a41188231 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:50.806850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998fce0-c7e2-4db6-8546-3f1a689af98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:50.807463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998fce0-c7e2-4db6-8546-3f1a689af98a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:50.807754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998fce0-c7e2-4db6-8546-3f1a689af98a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:50.808068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998fce0-c7e2-4db6-8546-3f1a689af98a None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:50.808738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998fce0-c7e2-4db6-8546-3f1a689af98a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:50.809042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998fce0-c7e2-4db6-8546-3f1a689af98a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:50.810442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998fce0-c7e2-4db6-8546-3f1a689af98a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:50.812853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabbe54-a00f-4979-8379-435a41188231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-6eabbe54-a00f-4979-8379-435a41188231 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:50.817978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:50.819191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:50.820157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 323/629] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:50.820889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b1165-1148-4148-8b12-1eccabc79749 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:50.820889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b1165-1148-4148-8b12-1eccabc79749 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:50.820889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b1165-1148-4148-8b12-1eccabc79749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:50.832631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 307/630] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:50.833622 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998fce0-c7e2-4db6-8546-3f1a689af98a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:50.845554 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-211b1165-1148-4148-8b12-1eccabc79749 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:50.846761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-211b1165-1148-4148-8b12-1eccabc79749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:50.913100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:50.931958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 324/631] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:50.931958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b1165-1148-4148-8b12-1eccabc79749 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:50.966881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998fce0-c7e2-4db6-8546-3f1a689af98a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:50.985392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:51.023918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57184fb4-cca3-48c9-a45d-47700ba83f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:51.024448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57184fb4-cca3-48c9-a45d-47700ba83f9e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:51.024732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57184fb4-cca3-48c9-a45d-47700ba83f9e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:51.025008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57184fb4-cca3-48c9-a45d-47700ba83f9e None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:51.025676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57184fb4-cca3-48c9-a45d-47700ba83f9e None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:51.025980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57184fb4-cca3-48c9-a45d-47700ba83f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:51.028080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57184fb4-cca3-48c9-a45d-47700ba83f9e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:51.042704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 308/632] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:51.049973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57184fb4-cca3-48c9-a45d-47700ba83f9e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:51.093202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b1165-1148-4148-8b12-1eccabc79749 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:51.140560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b5435-84e2-4c74-a15b-de3debbc848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:51.141213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b5435-84e2-4c74-a15b-de3debbc848d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:51.141498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b5435-84e2-4c74-a15b-de3debbc848d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:51.141788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b5435-84e2-4c74-a15b-de3debbc848d None admin] PATH_INFO: `/v3/users/glance` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:51.157837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b5435-84e2-4c74-a15b-de3debbc848d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:51.159294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b5435-84e2-4c74-a15b-de3debbc848d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-dd5b5435-84e2-4c74-a15b-de3debbc848d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:51.185335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:51.186389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:51.187399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:51.187399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:51.187399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:51.187399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:51.187399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:51.187399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:51.187399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:51.187399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:51.187399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:31:51.187399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:51.187399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 325/633] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:51.193374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b5435-84e2-4c74-a15b-de3debbc848d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:51.215942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57184fb4-cca3-48c9-a45d-47700ba83f9e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:51.253266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfbcea-8ffb-4e9f-8e52-443a404a26cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:51.254363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfbcea-8ffb-4e9f-8e52-443a404a26cc None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:51.254651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfbcea-8ffb-4e9f-8e52-443a404a26cc None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:51.254921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfbcea-8ffb-4e9f-8e52-443a404a26cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:51.260131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:51.276897 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfbcea-8ffb-4e9f-8e52-443a404a26cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:51.278742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfbcea-8ffb-4e9f-8e52-443a404a26cc None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-7bcfbcea-8ffb-4e9f-8e52-443a404a26cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:51.293596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:51.294624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:51.295537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:51.295537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:51.295537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:51.297610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 309/634] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:51.298678 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfbcea-8ffb-4e9f-8e52-443a404a26cc None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:51.354339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:51.360699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b5435-84e2-4c74-a15b-de3debbc848d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:51.404898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebf3f9-7f14-4a15-8f46-f99e820d5b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:51.405548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebf3f9-7f14-4a15-8f46-f99e820d5b5b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:51.405830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebf3f9-7f14-4a15-8f46-f99e820d5b5b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:51.406152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebf3f9-7f14-4a15-8f46-f99e820d5b5b None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:51.406683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfbcea-8ffb-4e9f-8e52-443a404a26cc None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:51.406862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebf3f9-7f14-4a15-8f46-f99e820d5b5b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:51.407167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebf3f9-7f14-4a15-8f46-f99e820d5b5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:51.408763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebf3f9-7f14-4a15-8f46-f99e820d5b5b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:51.435371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 326/635] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Aug 26 15:31:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:51.436289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebf3f9-7f14-4a15-8f46-f99e820d5b5b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:51.448356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000ab70-fbb9-4d23-841d-ad874bdf695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:51.449077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000ab70-fbb9-4d23-841d-ad874bdf695e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:51.449436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000ab70-fbb9-4d23-841d-ad874bdf695e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:51.449790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000ab70-fbb9-4d23-841d-ad874bdf695e None admin] PATH_INFO: `/v3/roles/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:51.461553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000ab70-fbb9-4d23-841d-ad874bdf695e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-d000ab70-fbb9-4d23-841d-ad874bdf695e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:51.466934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:51.476123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 310/636] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:51.477008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000ab70-fbb9-4d23-841d-ad874bdf695e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:51.527003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:51.571522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebf3f9-7f14-4a15-8f46-f99e820d5b5b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:51.600909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000ab70-fbb9-4d23-841d-ad874bdf695e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:51.612358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe9461-e5e0-41ba-af78-790e3014f630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:51.613096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe9461-e5e0-41ba-af78-790e3014f630 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:51.613483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe9461-e5e0-41ba-af78-790e3014f630 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:51.613866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe9461-e5e0-41ba-af78-790e3014f630 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:51.614615 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe9461-e5e0-41ba-af78-790e3014f630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:51.615161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe9461-e5e0-41ba-af78-790e3014f630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:51.617159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe9461-e5e0-41ba-af78-790e3014f630 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:51.629684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 327/637] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:51.630817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe9461-e5e0-41ba-af78-790e3014f630 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:51.645853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102c256-cb25-4680-b080-b7dedb74eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:51.646997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102c256-cb25-4680-b080-b7dedb74eb64 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:51.647442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102c256-cb25-4680-b080-b7dedb74eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:51.647900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102c256-cb25-4680-b080-b7dedb74eb64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:51.652333 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:51.732679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102c256-cb25-4680-b080-b7dedb74eb64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:51.734454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102c256-cb25-4680-b080-b7dedb74eb64 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:51.739800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe9461-e5e0-41ba-af78-790e3014f630 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:51.776233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0eedfa-84d1-492a-9c6d-521a58d062e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:51.776838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0eedfa-84d1-492a-9c6d-521a58d062e3 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:51.777176 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0eedfa-84d1-492a-9c6d-521a58d062e3 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:51.777397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0eedfa-84d1-492a-9c6d-521a58d062e3 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:51.778059 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0eedfa-84d1-492a-9c6d-521a58d062e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:51.778364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0eedfa-84d1-492a-9c6d-521a58d062e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:51.779736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0eedfa-84d1-492a-9c6d-521a58d062e3 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:51.792774 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 328/638] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:51.792774 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0eedfa-84d1-492a-9c6d-521a58d062e3 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:51.793942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:51.851397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 311/639] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:51.852697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f260c5-5026-4c39-9b10-33974848c797 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:51.853038 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f260c5-5026-4c39-9b10-33974848c797 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:51.853320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f260c5-5026-4c39-9b10-33974848c797 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:51.858659 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 312/640] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:51.858659 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f260c5-5026-4c39-9b10-33974848c797 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:51.888464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:51.910125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0eedfa-84d1-492a-9c6d-521a58d062e3 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:51.945052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86408c8-c0ad-4dde-bdbf-a9d500f95184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:51.945674 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86408c8-c0ad-4dde-bdbf-a9d500f95184 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:51.945957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86408c8-c0ad-4dde-bdbf-a9d500f95184 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:51.946234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86408c8-c0ad-4dde-bdbf-a9d500f95184 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:51.953366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86408c8-c0ad-4dde-bdbf-a9d500f95184 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:51.954821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86408c8-c0ad-4dde-bdbf-a9d500f95184 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:51.963592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f260c5-5026-4c39-9b10-33974848c797 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-f86408c8-c0ad-4dde-bdbf-a9d500f95184 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:51.966800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:51.967759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:51.968759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:51.968759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:51.968759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:51.968759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:51.968759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:51.968759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:51.968759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:51.968759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:51.968759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:31:51.968759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:51.968759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 329/641] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Aug 26 15:31:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:51.969398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86408c8-c0ad-4dde-bdbf-a9d500f95184 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:52.000131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88006c3-e08d-4515-87be-607d31e3a410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:52.000927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88006c3-e08d-4515-87be-607d31e3a410 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.001213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88006c3-e08d-4515-87be-607d31e3a410 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.001494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88006c3-e08d-4515-87be-607d31e3a410 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.012871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88006c3-e08d-4515-87be-607d31e3a410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:52.014717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88006c3-e08d-4515-87be-607d31e3a410 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:52.022665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-e88006c3-e08d-4515-87be-607d31e3a410 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:52.033190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:52.034239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:52.035216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:52.035216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:52.035216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:52.035216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 313/642] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:52.035216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b63946-fecd-437e-909f-99f337f53fab None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.035216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b63946-fecd-437e-909f-99f337f53fab None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.035216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b63946-fecd-437e-909f-99f337f53fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.050956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-18b63946-fecd-437e-909f-99f337f53fab None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:52.053460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-18b63946-fecd-437e-909f-99f337f53fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:52.081496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86408c8-c0ad-4dde-bdbf-a9d500f95184 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:52.121345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80059f78-444c-484e-9999-490ed704f054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:52.122021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80059f78-444c-484e-9999-490ed704f054 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.122312 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80059f78-444c-484e-9999-490ed704f054 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.122590 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80059f78-444c-484e-9999-490ed704f054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.132133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 314/643] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:52.132133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b63946-fecd-437e-909f-99f337f53fab None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:52.141765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:52.148427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80059f78-444c-484e-9999-490ed704f054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:52.151198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80059f78-444c-484e-9999-490ed704f054 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-80059f78-444c-484e-9999-490ed704f054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:52.171142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:52.172431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:52.173516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:52.173516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:52.173516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:52.174268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 330/644] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:52.175552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80059f78-444c-484e-9999-490ed704f054 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:52.236546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b63946-fecd-437e-909f-99f337f53fab None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:52.265968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:52.274187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa5cd8-ebce-4194-ab7c-8044cfe0acfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:52.274802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa5cd8-ebce-4194-ab7c-8044cfe0acfd None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.275089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa5cd8-ebce-4194-ab7c-8044cfe0acfd None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.275367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa5cd8-ebce-4194-ab7c-8044cfe0acfd None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.276128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa5cd8-ebce-4194-ab7c-8044cfe0acfd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:52.276441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa5cd8-ebce-4194-ab7c-8044cfe0acfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:52.277872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa5cd8-ebce-4194-ab7c-8044cfe0acfd None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:52.295615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 315/645] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Aug 26 15:31:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:52.296583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa5cd8-ebce-4194-ab7c-8044cfe0acfd None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:52.318196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80059f78-444c-484e-9999-490ed704f054 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:52.367445 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe951-6408-4489-b430-78a1469c1a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:52.368595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe951-6408-4489-b430-78a1469c1a8e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.369619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe951-6408-4489-b430-78a1469c1a8e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.370122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe951-6408-4489-b430-78a1469c1a8e None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.371304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fe951-6408-4489-b430-78a1469c1a8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:52.371987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fe951-6408-4489-b430-78a1469c1a8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:52.374381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fe951-6408-4489-b430-78a1469c1a8e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:52.390583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 331/646] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:52.392402 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe951-6408-4489-b430-78a1469c1a8e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:52.404710 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa5cd8-ebce-4194-ab7c-8044cfe0acfd None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:52.442216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb9752-e106-4627-a6d9-9fb8ccdbadbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:52.442830 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb9752-e106-4627-a6d9-9fb8ccdbadbf None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.443159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb9752-e106-4627-a6d9-9fb8ccdbadbf None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.443479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb9752-e106-4627-a6d9-9fb8ccdbadbf None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.444176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb9752-e106-4627-a6d9-9fb8ccdbadbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:52.444529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb9752-e106-4627-a6d9-9fb8ccdbadbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:52.445917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb9752-e106-4627-a6d9-9fb8ccdbadbf None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:52.458097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 316/647] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:52.459015 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb9752-e106-4627-a6d9-9fb8ccdbadbf None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:52.467174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:52.510368 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:52.516156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe951-6408-4489-b430-78a1469c1a8e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:52.553892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccab0e-eff9-4bcb-ab10-666da677176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:52.554418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccab0e-eff9-4bcb-ab10-666da677176d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.554705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccab0e-eff9-4bcb-ab10-666da677176d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.554984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccab0e-eff9-4bcb-ab10-666da677176d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.569709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb9752-e106-4627-a6d9-9fb8ccdbadbf None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:52.610969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24c2aa-6995-4a02-a2e9-b6f1cdc241b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:52.611938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24c2aa-6995-4a02-a2e9-b6f1cdc241b5 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.612394 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24c2aa-6995-4a02-a2e9-b6f1cdc241b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.612907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24c2aa-6995-4a02-a2e9-b6f1cdc241b5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.621640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24c2aa-6995-4a02-a2e9-b6f1cdc241b5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:52.623373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24c2aa-6995-4a02-a2e9-b6f1cdc241b5 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:52.631294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccab0e-eff9-4bcb-ab10-666da677176d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:52.632793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccab0e-eff9-4bcb-ab10-666da677176d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-ca24c2aa-6995-4a02-a2e9-b6f1cdc241b5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:52.635648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:52.636651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:52.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:52.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:52.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:52.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:52.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:52.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:52.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:52.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:52.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:31:52.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:52.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 317/648] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:52.638260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24c2aa-6995-4a02-a2e9-b6f1cdc241b5 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:52.684854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:52.734991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 332/649] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:52.735818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccab0e-eff9-4bcb-ab10-666da677176d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:52.740471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24c2aa-6995-4a02-a2e9-b6f1cdc241b5 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:52.748051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:52.774157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bc60b-c612-4b84-9c54-445d5a34b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:52.774789 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bc60b-c612-4b84-9c54-445d5a34b4d1 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.775068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bc60b-c612-4b84-9c54-445d5a34b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.775342 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bc60b-c612-4b84-9c54-445d5a34b4d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.786056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bc60b-c612-4b84-9c54-445d5a34b4d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-635bc60b-c612-4b84-9c54-445d5a34b4d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:52.791282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:52.792450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 318/650] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:52.793496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bc60b-c612-4b84-9c54-445d5a34b4d1 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:52.837575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccab0e-eff9-4bcb-ab10-666da677176d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:52.873012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773f729-0e9e-4a30-b34e-2ab5ff0ece89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:52.873876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773f729-0e9e-4a30-b34e-2ab5ff0ece89 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.874158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773f729-0e9e-4a30-b34e-2ab5ff0ece89 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.874436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773f729-0e9e-4a30-b34e-2ab5ff0ece89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.880962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773f729-0e9e-4a30-b34e-2ab5ff0ece89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:52.882403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773f729-0e9e-4a30-b34e-2ab5ff0ece89 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:52.891304 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bc60b-c612-4b84-9c54-445d5a34b4d1 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-b773f729-0e9e-4a30-b34e-2ab5ff0ece89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:52.893152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:52.894226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:52.895563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:52.895563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:52.895563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:52.895563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:52.895563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:52.895563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:52.895563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:52.895563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:52.895563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 333/651] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:52.895563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee50f3-24fa-4108-92d8-90caff216aad None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.896281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee50f3-24fa-4108-92d8-90caff216aad None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.896281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee50f3-24fa-4108-92d8-90caff216aad None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.897588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 334/652] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:52.899910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee50f3-24fa-4108-92d8-90caff216aad None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:52.910567 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:52.929490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7413e-39fd-4d10-99dc-3b0296b652fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:52.930312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7413e-39fd-4d10-99dc-3b0296b652fe None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:52.930413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7413e-39fd-4d10-99dc-3b0296b652fe None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:52.930665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7413e-39fd-4d10-99dc-3b0296b652fe None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:52.931387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7413e-39fd-4d10-99dc-3b0296b652fe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:52.931688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7413e-39fd-4d10-99dc-3b0296b652fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:52.933268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7413e-39fd-4d10-99dc-3b0296b652fe None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:52.950423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 319/653] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Aug 26 15:31:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:52.951442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7413e-39fd-4d10-99dc-3b0296b652fe None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:52.978110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:53.001230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee50f3-24fa-4108-92d8-90caff216aad None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:53.034523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55565f-5534-4cda-b1b2-4eca9ad074ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:53.035144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55565f-5534-4cda-b1b2-4eca9ad074ed None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:53.035428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55565f-5534-4cda-b1b2-4eca9ad074ed None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:53.035703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55565f-5534-4cda-b1b2-4eca9ad074ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:53.046452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55565f-5534-4cda-b1b2-4eca9ad074ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:53.047914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55565f-5534-4cda-b1b2-4eca9ad074ed None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:53.050797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7413e-39fd-4d10-99dc-3b0296b652fe None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-1b55565f-5534-4cda-b1b2-4eca9ad074ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:53.062865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:53.063988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:53.064912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:53.064912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:53.064912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:53.064912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 335/654] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:53.065141 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55565f-5534-4cda-b1b2-4eca9ad074ed None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:53.090819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0db91-8a32-4652-9584-9d206bd99b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:53.091654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0db91-8a32-4652-9584-9d206bd99b92 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:53.091972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0db91-8a32-4652-9584-9d206bd99b92 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:53.092255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0db91-8a32-4652-9584-9d206bd99b92 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:53.092927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0db91-8a32-4652-9584-9d206bd99b92 None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:53.093231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0db91-8a32-4652-9584-9d206bd99b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:53.094706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0db91-8a32-4652-9584-9d206bd99b92 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:53.103034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 320/655] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:53.104262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befacc1-d2fd-4956-9fe3-9b6ce8acfbba None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:53.104553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befacc1-d2fd-4956-9fe3-9b6ce8acfbba None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:53.104834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befacc1-d2fd-4956-9fe3-9b6ce8acfbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:53.121891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7befacc1-d2fd-4956-9fe3-9b6ce8acfbba None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:53.123072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7befacc1-d2fd-4956-9fe3-9b6ce8acfbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:53.126554 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:53.165166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55565f-5534-4cda-b1b2-4eca9ad074ed None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:53.189340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 321/656] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:53.190174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befacc1-d2fd-4956-9fe3-9b6ce8acfbba None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:53.206064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246bb78-51e1-4d54-9beb-8fa911d0f1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:53.206673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246bb78-51e1-4d54-9beb-8fa911d0f1b3 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:53.206959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246bb78-51e1-4d54-9beb-8fa911d0f1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:53.207238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246bb78-51e1-4d54-9beb-8fa911d0f1b3 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:53.207954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246bb78-51e1-4d54-9beb-8fa911d0f1b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:53.208272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246bb78-51e1-4d54-9beb-8fa911d0f1b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:53.208722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:53.210244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246bb78-51e1-4d54-9beb-8fa911d0f1b3 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:53.227262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 336/657] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:53.227262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246bb78-51e1-4d54-9beb-8fa911d0f1b3 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:53.302573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befacc1-d2fd-4956-9fe3-9b6ce8acfbba None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:53.327166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246bb78-51e1-4d54-9beb-8fa911d0f1b3 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:53.338476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24373651-baad-46ad-8a82-e27b41916793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:53.339077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24373651-baad-46ad-8a82-e27b41916793 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:53.339361 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24373651-baad-46ad-8a82-e27b41916793 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:53.339634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24373651-baad-46ad-8a82-e27b41916793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:53.352603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:53.365872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc68a7f-8561-4a2f-b207-6ac922ef5821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:53.366520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68a7f-8561-4a2f-b207-6ac922ef5821 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:53.367281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68a7f-8561-4a2f-b207-6ac922ef5821 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:53.367281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68a7f-8561-4a2f-b207-6ac922ef5821 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:53.367793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc68a7f-8561-4a2f-b207-6ac922ef5821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:53.368140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc68a7f-8561-4a2f-b207-6ac922ef5821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:53.369511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc68a7f-8561-4a2f-b207-6ac922ef5821 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:53.383257 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 337/658] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:53.384436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc68a7f-8561-4a2f-b207-6ac922ef5821 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:53.411462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24373651-baad-46ad-8a82-e27b41916793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:53.413473 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24373651-baad-46ad-8a82-e27b41916793 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:53.431440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:53.487295 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc68a7f-8561-4a2f-b207-6ac922ef5821 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:53.515976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 322/659] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:53.516819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24373651-baad-46ad-8a82-e27b41916793 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:53.520219 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d2ff9-d684-43ce-94a3-258b66c04b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:53.520854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2ff9-d684-43ce-94a3-258b66c04b57 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:53.521138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2ff9-d684-43ce-94a3-258b66c04b57 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:53.521932 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2ff9-d684-43ce-94a3-258b66c04b57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:53.533446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d2ff9-d684-43ce-94a3-258b66c04b57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:53.534887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d2ff9-d684-43ce-94a3-258b66c04b57 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-1c9d2ff9-d684-43ce-94a3-258b66c04b57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:53.549367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:53.550393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:53.551370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:53.551370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:53.551370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:53.551370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 338/660] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:53.551599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d2ff9-d684-43ce-94a3-258b66c04b57 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:53.591936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:53.623214 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24373651-baad-46ad-8a82-e27b41916793 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:53.638842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:53.656329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811986-d734-4b6b-870d-6e0e3676fc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:53.656930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811986-d734-4b6b-870d-6e0e3676fc90 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:53.657211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811986-d734-4b6b-870d-6e0e3676fc90 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:53.657487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811986-d734-4b6b-870d-6e0e3676fc90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:53.658213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72811986-d734-4b6b-870d-6e0e3676fc90 None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=cef193a54eff44f2b53716e3ec1a29cf) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:53.658520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72811986-d734-4b6b-870d-6e0e3676fc90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:53.659981 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72811986-d734-4b6b-870d-6e0e3676fc90 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:53.662521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d2ff9-d684-43ce-94a3-258b66c04b57 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:53.666451 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 323/661] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:53 2022] GET /identity/v3/role_assignments?user.id=cef193a54eff44f2b53716e3ec1a29cf&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:53.667346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811986-d734-4b6b-870d-6e0e3676fc90 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:53.713442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f4600-c92e-4a55-a5fe-6a3c78ab9929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:53.714107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f4600-c92e-4a55-a5fe-6a3c78ab9929 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:53.714391 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f4600-c92e-4a55-a5fe-6a3c78ab9929 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:53.714662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f4600-c92e-4a55-a5fe-6a3c78ab9929 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:53.724447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f4600-c92e-4a55-a5fe-6a3c78ab9929 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:53.727591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f4600-c92e-4a55-a5fe-6a3c78ab9929 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-1d3f4600-c92e-4a55-a5fe-6a3c78ab9929 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:53.744261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:53.745327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:53.746232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:53.746232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:53.746232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:53.746232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:53.746232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:53.746232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:53.746232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:53.746232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:53.746232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 339/662] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:53.746731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f4600-c92e-4a55-a5fe-6a3c78ab9929 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:53.794083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811986-d734-4b6b-870d-6e0e3676fc90 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:53.834397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdac67-8102-45b9-b8aa-513f878b8d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:53.835098 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdac67-8102-45b9-b8aa-513f878b8d5a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:53.835770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdac67-8102-45b9-b8aa-513f878b8d5a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:53.835770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdac67-8102-45b9-b8aa-513f878b8d5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:53.848840 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdac67-8102-45b9-b8aa-513f878b8d5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-dbbdac67-8102-45b9-b8aa-513f878b8d5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:53.854995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:53.856152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 324/663] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:53.857385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdac67-8102-45b9-b8aa-513f878b8d5a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:53.862135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:53.870666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f4600-c92e-4a55-a5fe-6a3c78ab9929 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:53.901841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:53.912212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e540c86-9a42-4f0a-9227-58991b96faf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:53.912841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e540c86-9a42-4f0a-9227-58991b96faf5 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:53.913125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e540c86-9a42-4f0a-9227-58991b96faf5 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:53.913401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e540c86-9a42-4f0a-9227-58991b96faf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:53.923693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e540c86-9a42-4f0a-9227-58991b96faf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:53.925383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e540c86-9a42-4f0a-9227-58991b96faf5 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-4e540c86-9a42-4f0a-9227-58991b96faf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:53.940510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:53.941666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:53.942725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:53.942725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:53.942725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:53.942978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 340/664] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:53.948345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e540c86-9a42-4f0a-9227-58991b96faf5 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:53.965805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdac67-8102-45b9-b8aa-513f878b8d5a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.000768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68ebb4-e6b6-4941-a040-3a3d5ea558a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:54.001469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68ebb4-e6b6-4941-a040-3a3d5ea558a2 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:54.001755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68ebb4-e6b6-4941-a040-3a3d5ea558a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:54.002034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68ebb4-e6b6-4941-a040-3a3d5ea558a2 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:54.002687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68ebb4-e6b6-4941-a040-3a3d5ea558a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:54.002992 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68ebb4-e6b6-4941-a040-3a3d5ea558a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:54.004416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68ebb4-e6b6-4941-a040-3a3d5ea558a2 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:54.027573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 325/665] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:54.028812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68ebb4-e6b6-4941-a040-3a3d5ea558a2 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:54.057410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e540c86-9a42-4f0a-9227-58991b96faf5 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.097273 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f61055-b1ec-4b20-a218-1c0e0489cca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:54.097885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f61055-b1ec-4b20-a218-1c0e0489cca0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:54.098167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f61055-b1ec-4b20-a218-1c0e0489cca0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:54.098441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f61055-b1ec-4b20-a218-1c0e0489cca0 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:54.099377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f61055-b1ec-4b20-a218-1c0e0489cca0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:54.099682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f61055-b1ec-4b20-a218-1c0e0489cca0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:54.101154 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f61055-b1ec-4b20-a218-1c0e0489cca0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:54.121326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 341/666] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:54.121326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f61055-b1ec-4b20-a218-1c0e0489cca0 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:54.132017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:54.155378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:54.158743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68ebb4-e6b6-4941-a040-3a3d5ea558a2 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.194742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556477b-3a8e-4bc3-a03a-b20012682903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:54.195386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556477b-3a8e-4bc3-a03a-b20012682903 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:54.195671 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556477b-3a8e-4bc3-a03a-b20012682903 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:54.195978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556477b-3a8e-4bc3-a03a-b20012682903 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:54.196649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556477b-3a8e-4bc3-a03a-b20012682903 None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:54.196958 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556477b-3a8e-4bc3-a03a-b20012682903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:54.198400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556477b-3a8e-4bc3-a03a-b20012682903 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:54.212185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 326/667] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:31:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:54.212185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556477b-3a8e-4bc3-a03a-b20012682903 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:54.236166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f61055-b1ec-4b20-a218-1c0e0489cca0 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.273892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf5766-bbd3-462c-bc4a-5fe6923ea042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:54.274494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf5766-bbd3-462c-bc4a-5fe6923ea042 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:54.274773 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf5766-bbd3-462c-bc4a-5fe6923ea042 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:54.275045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf5766-bbd3-462c-bc4a-5fe6923ea042 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:54.275680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf5766-bbd3-462c-bc4a-5fe6923ea042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:54.276018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf5766-bbd3-462c-bc4a-5fe6923ea042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:54.277368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf5766-bbd3-462c-bc4a-5fe6923ea042 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:54.289634 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 342/668] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:54.290429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf5766-bbd3-462c-bc4a-5fe6923ea042 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:54.319625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556477b-3a8e-4bc3-a03a-b20012682903 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.355460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12afbc-63d1-47f2-9d0a-7fa8a2d2d923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:54.356287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12afbc-63d1-47f2-9d0a-7fa8a2d2d923 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:54.356879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12afbc-63d1-47f2-9d0a-7fa8a2d2d923 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:54.357171 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12afbc-63d1-47f2-9d0a-7fa8a2d2d923 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:54.360795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:54.365169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12afbc-63d1-47f2-9d0a-7fa8a2d2d923 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:54.366621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12afbc-63d1-47f2-9d0a-7fa8a2d2d923 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-7c12afbc-63d1-47f2-9d0a-7fa8a2d2d923 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:54.378191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:54.379213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 327/669] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:31:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:54.380267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12afbc-63d1-47f2-9d0a-7fa8a2d2d923 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:54.399774 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf5766-bbd3-462c-bc4a-5fe6923ea042 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.410000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:54.438581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a5a2d-83cd-4da1-a44d-9d9fc54739bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:54.439597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a5a2d-83cd-4da1-a44d-9d9fc54739bf None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:54.440083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a5a2d-83cd-4da1-a44d-9d9fc54739bf None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:54.440535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a5a2d-83cd-4da1-a44d-9d9fc54739bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:54.441440 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a5a2d-83cd-4da1-a44d-9d9fc54739bf None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=d1b01c21c38949938141eefdc4939385) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:54.441959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a5a2d-83cd-4da1-a44d-9d9fc54739bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:54.443580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a5a2d-83cd-4da1-a44d-9d9fc54739bf None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:54.450292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 343/670] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:54 2022] GET /identity/v3/role_assignments?user.id=d1b01c21c38949938141eefdc4939385&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:54.451262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a5a2d-83cd-4da1-a44d-9d9fc54739bf None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:54.504085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12afbc-63d1-47f2-9d0a-7fa8a2d2d923 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.557682 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228600cd-f7b3-4403-b6f6-6d60a80e587a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:54.558333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228600cd-f7b3-4403-b6f6-6d60a80e587a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:54.558619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228600cd-f7b3-4403-b6f6-6d60a80e587a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:54.558897 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228600cd-f7b3-4403-b6f6-6d60a80e587a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:54.576987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228600cd-f7b3-4403-b6f6-6d60a80e587a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:54.578760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228600cd-f7b3-4403-b6f6-6d60a80e587a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:54.587258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a5a2d-83cd-4da1-a44d-9d9fc54739bf None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-228600cd-f7b3-4403-b6f6-6d60a80e587a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:54.601456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:54.602511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:54.603430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:54.603430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:54.603430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:54.610131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 328/671] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:54.610131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228600cd-f7b3-4403-b6f6-6d60a80e587a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:54.635598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:54.645232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e8a1d-a92a-44fa-b179-85bf9995f2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:54.648606 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e8a1d-a92a-44fa-b179-85bf9995f2b6 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:54.648606 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e8a1d-a92a-44fa-b179-85bf9995f2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:54.648606 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e8a1d-a92a-44fa-b179-85bf9995f2b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:54.657457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e8a1d-a92a-44fa-b179-85bf9995f2b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:54.658983 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e8a1d-a92a-44fa-b179-85bf9995f2b6 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-d11e8a1d-a92a-44fa-b179-85bf9995f2b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:54.671731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:54.672890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:54.673974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:54.673974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:54.673974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:54.673974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:54.673974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:54.673974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:54.673974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:54.673974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:54.673974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 344/672] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:54.674562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e8a1d-a92a-44fa-b179-85bf9995f2b6 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:54.720264 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:54.754329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228600cd-f7b3-4403-b6f6-6d60a80e587a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.797268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20324619-200d-4455-98fa-2293baeff157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:54.797909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20324619-200d-4455-98fa-2293baeff157 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:54.798198 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20324619-200d-4455-98fa-2293baeff157 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:54.798741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20324619-200d-4455-98fa-2293baeff157 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:54.800016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20324619-200d-4455-98fa-2293baeff157 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:54.800330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20324619-200d-4455-98fa-2293baeff157 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:54.802788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20324619-200d-4455-98fa-2293baeff157 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:54.820401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e8a1d-a92a-44fa-b179-85bf9995f2b6 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.827458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 329/673] 10.222.0.98 () {56 vars in 1139 bytes} [Fri Aug 26 15:31:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:54.832453 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20324619-200d-4455-98fa-2293baeff157 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:54.865416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83702ff0-9ac0-4fa0-99f6-0aaa4675d161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:54.866101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83702ff0-9ac0-4fa0-99f6-0aaa4675d161 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:54.866389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83702ff0-9ac0-4fa0-99f6-0aaa4675d161 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:54.866712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83702ff0-9ac0-4fa0-99f6-0aaa4675d161 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:54.867381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83702ff0-9ac0-4fa0-99f6-0aaa4675d161 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:54.867686 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83702ff0-9ac0-4fa0-99f6-0aaa4675d161 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:54.869270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83702ff0-9ac0-4fa0-99f6-0aaa4675d161 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:54.883501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 345/674] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:54.884498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83702ff0-9ac0-4fa0-99f6-0aaa4675d161 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:54.937030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:54.953523 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20324619-200d-4455-98fa-2293baeff157 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.988018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:54.991510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83702ff0-9ac0-4fa0-99f6-0aaa4675d161 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:54.994323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b46ebe-b065-43e8-ac47-7c071c409ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:54.995030 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b46ebe-b065-43e8-ac47-7c071c409ecd None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:54.995321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b46ebe-b065-43e8-ac47-7c071c409ecd None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:54.995604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b46ebe-b065-43e8-ac47-7c071c409ecd None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:55.000369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b46ebe-b065-43e8-ac47-7c071c409ecd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:55.000369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b46ebe-b065-43e8-ac47-7c071c409ecd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:55.000369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b46ebe-b065-43e8-ac47-7c071c409ecd None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:55.011862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 330/675] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:55.012618 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b46ebe-b065-43e8-ac47-7c071c409ecd None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:55.033162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e6383-255c-43af-a869-3578e74224ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:55.033876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6383-255c-43af-a869-3578e74224ed None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:55.034166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6383-255c-43af-a869-3578e74224ed None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:55.034441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6383-255c-43af-a869-3578e74224ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:55.107093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e6383-255c-43af-a869-3578e74224ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:55.108790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e6383-255c-43af-a869-3578e74224ed None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:55.121116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b46ebe-b065-43e8-ac47-7c071c409ecd None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:55.160498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e149331-2d48-4c36-9bf4-b61f3ee2bfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:55.161216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e149331-2d48-4c36-9bf4-b61f3ee2bfc7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:55.161730 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e149331-2d48-4c36-9bf4-b61f3ee2bfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:55.162014 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e149331-2d48-4c36-9bf4-b61f3ee2bfc7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:55.171002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e149331-2d48-4c36-9bf4-b61f3ee2bfc7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:55.172511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e149331-2d48-4c36-9bf4-b61f3ee2bfc7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-7e149331-2d48-4c36-9bf4-b61f3ee2bfc7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:55.189295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:55.190294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:55.191252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:55.191252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:55.191252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:55.191252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:55.191252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:55.191252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:55.191252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:55.191252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:55.191252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:31:55.191252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:55.199021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 331/676] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:31:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:55.200327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e149331-2d48-4c36-9bf4-b61f3ee2bfc7 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:55.257211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:55.284362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:55.306329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e149331-2d48-4c36-9bf4-b61f3ee2bfc7 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:55.334636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 346/677] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:55.335566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e6383-255c-43af-a869-3578e74224ed None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:55.342047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774f477-8081-4d30-8d1c-607551ce71cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:55.343287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774f477-8081-4d30-8d1c-607551ce71cc None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:55.343797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774f477-8081-4d30-8d1c-607551ce71cc None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:55.344367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774f477-8081-4d30-8d1c-607551ce71cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:55.345284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774f477-8081-4d30-8d1c-607551ce71cc None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=3a1ff8aff9e048c6972843e0b50c4d45) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:55.345785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774f477-8081-4d30-8d1c-607551ce71cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:55.347387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774f477-8081-4d30-8d1c-607551ce71cc None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:55.354042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 332/678] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:55 2022] GET /identity/v3/role_assignments?user.id=3a1ff8aff9e048c6972843e0b50c4d45&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:55.355037 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774f477-8081-4d30-8d1c-607551ce71cc None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:55.478292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e6383-255c-43af-a869-3578e74224ed None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:55.499059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774f477-8081-4d30-8d1c-607551ce71cc None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:55.515086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94872b5e-bcf0-4298-80b1-7095d7124d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:55.515690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94872b5e-bcf0-4298-80b1-7095d7124d3a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:55.516006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94872b5e-bcf0-4298-80b1-7095d7124d3a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:55.516282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94872b5e-bcf0-4298-80b1-7095d7124d3a None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:55.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94872b5e-bcf0-4298-80b1-7095d7124d3a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:55.517315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94872b5e-bcf0-4298-80b1-7095d7124d3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:55.518740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94872b5e-bcf0-4298-80b1-7095d7124d3a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:55.535754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:55.538505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 347/679] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Aug 26 15:31:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:55.544028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94872b5e-bcf0-4298-80b1-7095d7124d3a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:55.565139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:55.566006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78a5f0-7d5c-455b-b636-b227be023d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:55.566760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78a5f0-7d5c-455b-b636-b227be023d5c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:55.567058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78a5f0-7d5c-455b-b636-b227be023d5c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:55.567340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78a5f0-7d5c-455b-b636-b227be023d5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:55.593045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78a5f0-7d5c-455b-b636-b227be023d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:55.597390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78a5f0-7d5c-455b-b636-b227be023d5c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-9c78a5f0-7d5c-455b-b636-b227be023d5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:55.633020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:55.634074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:55.634999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:55.634999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:55.634999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:55.638529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 333/680] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:55.641289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78a5f0-7d5c-455b-b636-b227be023d5c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:55.689840 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94872b5e-bcf0-4298-80b1-7095d7124d3a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:55.728069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7376199-f11a-42ed-94b2-072427f6b6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:55.728747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7376199-f11a-42ed-94b2-072427f6b6db None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:55.728971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7376199-f11a-42ed-94b2-072427f6b6db None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:55.729250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7376199-f11a-42ed-94b2-072427f6b6db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:55.767390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78a5f0-7d5c-455b-b636-b227be023d5c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:55.807050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa8493-88ff-4ef5-b9b8-e093a770a12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:55.807729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa8493-88ff-4ef5-b9b8-e093a770a12e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:55.808042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa8493-88ff-4ef5-b9b8-e093a770a12e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:55.808317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa8493-88ff-4ef5-b9b8-e093a770a12e None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:55.809022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa8493-88ff-4ef5-b9b8-e093a770a12e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:55.809324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa8493-88ff-4ef5-b9b8-e093a770a12e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:55.810754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa8493-88ff-4ef5-b9b8-e093a770a12e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:55.823887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 334/681] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:55.824925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7376199-f11a-42ed-94b2-072427f6b6db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:55.825233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d50bf-d8f7-485e-a58a-67f7d3eac7b7 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:55.825519 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d50bf-d8f7-485e-a58a-67f7d3eac7b7 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:55.825801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d50bf-d8f7-485e-a58a-67f7d3eac7b7 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:55.826656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7376199-f11a-42ed-94b2-072427f6b6db None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:55.827261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 335/682] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:55.829984 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d50bf-d8f7-485e-a58a-67f7d3eac7b7 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:55.844241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:55.867512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:55.942917 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 348/683] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 15:31:55.944806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd66572-8d87-40aa-9e8b-c78b8a8eaf2a None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:55.945212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd66572-8d87-40aa-9e8b-c78b8a8eaf2a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:55.945494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd66572-8d87-40aa-9e8b-c78b8a8eaf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:55.969330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-7bd66572-8d87-40aa-9e8b-c78b8a8eaf2a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:55.969989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d50bf-d8f7-485e-a58a-67f7d3eac7b7 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:55.970541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-7bd66572-8d87-40aa-9e8b-c78b8a8eaf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:56.013616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bf494-ab26-42fe-a6e6-b66c0ba05cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:56.014293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bf494-ab26-42fe-a6e6-b66c0ba05cc7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:56.014648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bf494-ab26-42fe-a6e6-b66c0ba05cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:56.015087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bf494-ab26-42fe-a6e6-b66c0ba05cc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:56.024688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bf494-ab26-42fe-a6e6-b66c0ba05cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:56.026670 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bf494-ab26-42fe-a6e6-b66c0ba05cc7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-dc3bf494-ab26-42fe-a6e6-b66c0ba05cc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:56.043487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:56.044489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:56.045418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:56.045418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:56.045418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:56.045418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:56.045418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:56.045418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:56.045418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:56.045418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:56.047959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 336/684] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:56.048884 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bf494-ab26-42fe-a6e6-b66c0ba05cc7 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:56.059790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 349/685] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:56.061308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd66572-8d87-40aa-9e8b-c78b8a8eaf2a None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:56.128306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:56.207882 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:56.228639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bf494-ab26-42fe-a6e6-b66c0ba05cc7 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:56.233856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd66572-8d87-40aa-9e8b-c78b8a8eaf2a None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:56.270415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540bd16-3818-40a8-b888-4626f81abe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:56.271504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540bd16-3818-40a8-b888-4626f81abe92 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:56.271786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540bd16-3818-40a8-b888-4626f81abe92 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:56.272086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540bd16-3818-40a8-b888-4626f81abe92 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:56.272784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540bd16-3818-40a8-b888-4626f81abe92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:56.273089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540bd16-3818-40a8-b888-4626f81abe92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:56.274523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540bd16-3818-40a8-b888-4626f81abe92 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:56.288696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 350/686] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:56.294472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540bd16-3818-40a8-b888-4626f81abe92 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:56.302308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1258a-5f49-433c-be2a-24f76819ed9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:56.303269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1258a-5f49-433c-be2a-24f76819ed9f None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:56.303754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1258a-5f49-433c-be2a-24f76819ed9f None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:56.304256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1258a-5f49-433c-be2a-24f76819ed9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:56.325797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1258a-5f49-433c-be2a-24f76819ed9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:56.327543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1258a-5f49-433c-be2a-24f76819ed9f None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-36b1258a-5f49-433c-be2a-24f76819ed9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:56.353841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:56.355474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:56.356617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:56.356617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:56.356617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:56.359088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 337/687] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:56.364790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1258a-5f49-433c-be2a-24f76819ed9f None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:56.406358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:56.410425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540bd16-3818-40a8-b888-4626f81abe92 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:56.452430 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a0ea1-3253-4009-a6e7-1e5e271b66f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:56.453035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a0ea1-3253-4009-a6e7-1e5e271b66f4 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:56.453325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a0ea1-3253-4009-a6e7-1e5e271b66f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:56.453632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a0ea1-3253-4009-a6e7-1e5e271b66f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:56.454369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a0ea1-3253-4009-a6e7-1e5e271b66f4 None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=8c6cad18359143868276eb9d79a3fd3b) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:56.454679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a0ea1-3253-4009-a6e7-1e5e271b66f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:56.456184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a0ea1-3253-4009-a6e7-1e5e271b66f4 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:56.472098 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 351/688] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:56 2022] GET /identity/v3/role_assignments?user.id=8c6cad18359143868276eb9d79a3fd3b&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:56.473150 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a0ea1-3253-4009-a6e7-1e5e271b66f4 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:56.510427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1258a-5f49-433c-be2a-24f76819ed9f None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:56.553210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433847-be4a-4b9b-967a-96d482faad63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:56.554532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433847-be4a-4b9b-967a-96d482faad63 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:56.555498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433847-be4a-4b9b-967a-96d482faad63 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:56.556010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433847-be4a-4b9b-967a-96d482faad63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:56.566980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:56.572203 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61433847-be4a-4b9b-967a-96d482faad63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:56.574152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61433847-be4a-4b9b-967a-96d482faad63 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-61433847-be4a-4b9b-967a-96d482faad63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:56.591892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:56.592928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:56.593875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:56.593875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:56.593875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:56.596409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 338/689] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:56.607099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433847-be4a-4b9b-967a-96d482faad63 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:56.728556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a0ea1-3253-4009-a6e7-1e5e271b66f4 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:56.739790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433847-be4a-4b9b-967a-96d482faad63 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:56.775457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6af97f-ee86-4a2d-9fc0-162800756813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:56.776159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6af97f-ee86-4a2d-9fc0-162800756813 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:56.776451 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6af97f-ee86-4a2d-9fc0-162800756813 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:56.776729 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6af97f-ee86-4a2d-9fc0-162800756813 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:56.777397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6af97f-ee86-4a2d-9fc0-162800756813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:56.777729 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6af97f-ee86-4a2d-9fc0-162800756813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:56.779112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6af97f-ee86-4a2d-9fc0-162800756813 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:56.793472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 352/690] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:56.794397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654036c5-6792-4966-ac69-a12d5577fe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:56.794763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9389de0-a00f-441d-8676-460304e12a9d None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:56.795126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9389de0-a00f-441d-8676-460304e12a9d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:56.795492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9389de0-a00f-441d-8676-460304e12a9d None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:56.796150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654036c5-6792-4966-ac69-a12d5577fe74 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:56.796614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654036c5-6792-4966-ac69-a12d5577fe74 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:56.797087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654036c5-6792-4966-ac69-a12d5577fe74 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:56.797191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 353/691] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:56.798051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654036c5-6792-4966-ac69-a12d5577fe74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:56.798466 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654036c5-6792-4966-ac69-a12d5577fe74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:56.802723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654036c5-6792-4966-ac69-a12d5577fe74 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:56.803380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9389de0-a00f-441d-8676-460304e12a9d None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:56.821703 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 339/692] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:56.823324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560f69c-b80b-4c16-ae05-ac956f3d6024 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:56.823617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560f69c-b80b-4c16-ae05-ac956f3d6024 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:56.823936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560f69c-b80b-4c16-ae05-ac956f3d6024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:56.837866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:56.847712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7560f69c-b80b-4c16-ae05-ac956f3d6024 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:56.849005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7560f69c-b80b-4c16-ae05-ac956f3d6024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:56.914497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:56.923268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9389de0-a00f-441d-8676-460304e12a9d None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:56.941562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 340/693] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:56.942436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560f69c-b80b-4c16-ae05-ac956f3d6024 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:56.991546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09c964-807c-4d38-9c31-e993918c2515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:56.992217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09c964-807c-4d38-9c31-e993918c2515 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:56.992499 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09c964-807c-4d38-9c31-e993918c2515 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:56.992776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09c964-807c-4d38-9c31-e993918c2515 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:57.006421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09c964-807c-4d38-9c31-e993918c2515 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:57.007902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09c964-807c-4d38-9c31-e993918c2515 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-4f09c964-807c-4d38-9c31-e993918c2515 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:31:57.025112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:57.026064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:57.026990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:31:57.026990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:31:57.026990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:31:57.026990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:31:57.026990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:31:57.026990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:31:57.026990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:31:57.026990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:57.032552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 354/694] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:57.038554 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09c964-807c-4d38-9c31-e993918c2515 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:57.055169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560f69c-b80b-4c16-ae05-ac956f3d6024 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:57.096559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9baaa9-2b3b-429b-9432-72728c8095ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:57.097445 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9baaa9-2b3b-429b-9432-72728c8095ca None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:57.097831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9baaa9-2b3b-429b-9432-72728c8095ca None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:57.098111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9baaa9-2b3b-429b-9432-72728c8095ca None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:57.099393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9baaa9-2b3b-429b-9432-72728c8095ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:57.100891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9baaa9-2b3b-429b-9432-72728c8095ca None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:57.160923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:57.179788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09c964-807c-4d38-9c31-e993918c2515 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:57.201567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 341/695] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:31:56 2022] POST /identity/v3/users => generated 254 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:31:57.215903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426ce35-46ae-479e-86cb-46cf2fa5fc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:57.216559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426ce35-46ae-479e-86cb-46cf2fa5fc54 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:57.216843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426ce35-46ae-479e-86cb-46cf2fa5fc54 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:57.217119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426ce35-46ae-479e-86cb-46cf2fa5fc54 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:57.217831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426ce35-46ae-479e-86cb-46cf2fa5fc54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:57.218137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426ce35-46ae-479e-86cb-46cf2fa5fc54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:57.219587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426ce35-46ae-479e-86cb-46cf2fa5fc54 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:57.233662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 355/696] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:31:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:57.240447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9baaa9-2b3b-429b-9432-72728c8095ca None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:57.309167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:57.394023 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9baaa9-2b3b-429b-9432-72728c8095ca None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:57.420860 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426ce35-46ae-479e-86cb-46cf2fa5fc54 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:57.431589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a8311-bc73-40cb-8c00-dc13c38a8f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:57.432329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a8311-bc73-40cb-8c00-dc13c38a8f41 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:57.432613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a8311-bc73-40cb-8c00-dc13c38a8f41 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:57.432893 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a8311-bc73-40cb-8c00-dc13c38a8f41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:57.433635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a8311-bc73-40cb-8c00-dc13c38a8f41 None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=1fae47d6b7a7472a9d94636bb126f802) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:57.433945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a8311-bc73-40cb-8c00-dc13c38a8f41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:57.435379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a8311-bc73-40cb-8c00-dc13c38a8f41 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:57.444566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 342/697] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:31:57 2022] GET /identity/v3/role_assignments?user.id=1fae47d6b7a7472a9d94636bb126f802&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:57.544536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426ce35-46ae-479e-86cb-46cf2fa5fc54 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:57.591017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323f7e5-ecb4-48cc-acce-eac7f4df1b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:57.591605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323f7e5-ecb4-48cc-acce-eac7f4df1b87 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:57.591897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323f7e5-ecb4-48cc-acce-eac7f4df1b87 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:57.592181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323f7e5-ecb4-48cc-acce-eac7f4df1b87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:57.605319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323f7e5-ecb4-48cc-acce-eac7f4df1b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-b323f7e5-ecb4-48cc-acce-eac7f4df1b87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:31:57.614903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:57.616362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:57.617313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 356/698] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:31:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:57.624473 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a8311-bc73-40cb-8c00-dc13c38a8f41 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:57.652107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5418de-2ccd-4103-bcf3-1daa337effa0 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:57.652442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5418de-2ccd-4103-bcf3-1daa337effa0 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:57.652665 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5418de-2ccd-4103-bcf3-1daa337effa0 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:57.655063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 357/699] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:31:57.660381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11335aa-8985-4b5d-8d1a-b559de8b466c None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:57.660666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11335aa-8985-4b5d-8d1a-b559de8b466c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:57.661472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11335aa-8985-4b5d-8d1a-b559de8b466c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:57.669174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:57.685372 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f11335aa-8985-4b5d-8d1a-b559de8b466c None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:57.686585 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f11335aa-8985-4b5d-8d1a-b559de8b466c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:57.759675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a8311-bc73-40cb-8c00-dc13c38a8f41 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:57.761494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:57.773308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 358/700] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:31:57.800762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c5cee-0f53-4251-a713-26ea3081660c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:57.801942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c5cee-0f53-4251-a713-26ea3081660c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:57.802429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c5cee-0f53-4251-a713-26ea3081660c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:57.802857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c5cee-0f53-4251-a713-26ea3081660c None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:57.803691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c5cee-0f53-4251-a713-26ea3081660c None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:57.804223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c5cee-0f53-4251-a713-26ea3081660c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:57.805787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c5cee-0f53-4251-a713-26ea3081660c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:57.815337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 343/701] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:31:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:57.821456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11335aa-8985-4b5d-8d1a-b559de8b466c None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:57.937681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11335aa-8985-4b5d-8d1a-b559de8b466c None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:57.973495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:57.978955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3891c0f-7800-47c8-a30c-c94dab09c334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:57.979599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3891c0f-7800-47c8-a30c-c94dab09c334 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:57.979949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3891c0f-7800-47c8-a30c-c94dab09c334 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:57.980238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3891c0f-7800-47c8-a30c-c94dab09c334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:57.991220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3891c0f-7800-47c8-a30c-c94dab09c334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:57.992756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3891c0f-7800-47c8-a30c-c94dab09c334 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-a3891c0f-7800-47c8-a30c-c94dab09c334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:58.015224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:58.016310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:58.017272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:58.017272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:58.017272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:58.017272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 359/702] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:58.022631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c5cee-0f53-4251-a713-26ea3081660c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:58.060870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3891c0f-7800-47c8-a30c-c94dab09c334 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:58.131963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c5cee-0f53-4251-a713-26ea3081660c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:58.168161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c57a01-a201-4da7-8974-f3fd8ecbf5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:58.168860 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57a01-a201-4da7-8974-f3fd8ecbf5ad None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:58.169145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57a01-a201-4da7-8974-f3fd8ecbf5ad None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:58.169422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57a01-a201-4da7-8974-f3fd8ecbf5ad None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:58.170074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c57a01-a201-4da7-8974-f3fd8ecbf5ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:58.170382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c57a01-a201-4da7-8974-f3fd8ecbf5ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:58.171757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c57a01-a201-4da7-8974-f3fd8ecbf5ad None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:58.184794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 344/703] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:58.190080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c57a01-a201-4da7-8974-f3fd8ecbf5ad None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:58.207534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3891c0f-7800-47c8-a30c-c94dab09c334 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:58.236311 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:58.308644 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c57a01-a201-4da7-8974-f3fd8ecbf5ad None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:58.327752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668d992-fb00-4f37-a25f-d20a77fd1ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:58.328453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668d992-fb00-4f37-a25f-d20a77fd1ca8 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:58.328737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668d992-fb00-4f37-a25f-d20a77fd1ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:58.329016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668d992-fb00-4f37-a25f-d20a77fd1ca8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:58.329608 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d668d992-fb00-4f37-a25f-d20a77fd1ca8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:58.330986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d668d992-fb00-4f37-a25f-d20a77fd1ca8 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:58.354421 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ad64-ed1c-4aaf-be37-24983cbf7e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:58.355055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ad64-ed1c-4aaf-be37-24983cbf7e82 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:58.355338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ad64-ed1c-4aaf-be37-24983cbf7e82 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:58.355615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ad64-ed1c-4aaf-be37-24983cbf7e82 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-d668d992-fb00-4f37-a25f-d20a77fd1ca8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:58.361296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:58.362416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:31:58.363276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:58.366142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 360/704] 10.222.0.98 () {56 vars in 1129 bytes} [Fri Aug 26 15:31:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:31:58.367163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619ad64-ed1c-4aaf-be37-24983cbf7e82 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:58.368688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619ad64-ed1c-4aaf-be37-24983cbf7e82 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:58.371602 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668d992-fb00-4f37-a25f-d20a77fd1ca8 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-0619ad64-ed1c-4aaf-be37-24983cbf7e82 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:31:58.388881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:58.391378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:58.392646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:58.392646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:58.392646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:31:58.392646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:31:58.392646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:31:58.392646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:31:58.392646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:31:58.392646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:31:58.392646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:31:58.392646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:58.392646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 345/705] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Aug 26 15:31:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:58.396874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ad64-ed1c-4aaf-be37-24983cbf7e82 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:58.473340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:58.564389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:58.575430 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668d992-fb00-4f37-a25f-d20a77fd1ca8 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:58.595503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ad64-ed1c-4aaf-be37-24983cbf7e82 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:58.637519 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba95a2-df85-4552-a41b-8edcc19ae3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:58.638137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba95a2-df85-4552-a41b-8edcc19ae3b0 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:58.638422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba95a2-df85-4552-a41b-8edcc19ae3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:58.638702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba95a2-df85-4552-a41b-8edcc19ae3b0 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:58.639440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba95a2-df85-4552-a41b-8edcc19ae3b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:58.639752 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba95a2-df85-4552-a41b-8edcc19ae3b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:58.641550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba95a2-df85-4552-a41b-8edcc19ae3b0 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:58.661190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb1b2b-1c7c-443d-8b4f-b7338420b4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:58.661513 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 346/706] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Aug 26 15:31:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:58.662266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1b2b-1c7c-443d-8b4f-b7338420b4da None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:58.662266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1b2b-1c7c-443d-8b4f-b7338420b4da None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:58.662524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1b2b-1c7c-443d-8b4f-b7338420b4da None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:58.663132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb1b2b-1c7c-443d-8b4f-b7338420b4da None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:58.663443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb1b2b-1c7c-443d-8b4f-b7338420b4da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:58.668127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb1b2b-1c7c-443d-8b4f-b7338420b4da None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:58.669195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba95a2-df85-4552-a41b-8edcc19ae3b0 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:58.680005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 361/707] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Aug 26 15:31:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:58.688552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb1b2b-1c7c-443d-8b4f-b7338420b4da None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:58.857895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba95a2-df85-4552-a41b-8edcc19ae3b0 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:58.876968 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb1b2b-1c7c-443d-8b4f-b7338420b4da None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:58.900520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:58.923483 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd2a26-055b-4a6c-9553-fbd6eacba275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:58.924348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd2a26-055b-4a6c-9553-fbd6eacba275 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:58.924775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd2a26-055b-4a6c-9553-fbd6eacba275 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:58.925198 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd2a26-055b-4a6c-9553-fbd6eacba275 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:58.942701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577143c-c0aa-40cf-b00e-499af8d9981f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:58.943557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577143c-c0aa-40cf-b00e-499af8d9981f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:58.944056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577143c-c0aa-40cf-b00e-499af8d9981f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:58.944528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577143c-c0aa-40cf-b00e-499af8d9981f None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:58.946327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577143c-c0aa-40cf-b00e-499af8d9981f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:58.946787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577143c-c0aa-40cf-b00e-499af8d9981f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:58.948314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:58.949430 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577143c-c0aa-40cf-b00e-499af8d9981f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:58.959996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 362/708] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Aug 26 15:31:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:31:59.014525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400c9c-6f05-46da-ad48-2c7a5100a8f5 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:59.014991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400c9c-6f05-46da-ad48-2c7a5100a8f5 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:59.015375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400c9c-6f05-46da-ad48-2c7a5100a8f5 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:59.017228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 363/709] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:59.023569 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddfbcf-0d7c-42ff-8250-e59886be3bbf None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:59.023880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddfbcf-0d7c-42ff-8250-e59886be3bbf None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:59.024170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddfbcf-0d7c-42ff-8250-e59886be3bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:59.042248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd2a26-055b-4a6c-9553-fbd6eacba275 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:59.043414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-48ddfbcf-0d7c-42ff-8250-e59886be3bbf None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:59.043755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd2a26-055b-4a6c-9553-fbd6eacba275 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:59.044700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-48ddfbcf-0d7c-42ff-8250-e59886be3bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:59.150644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 364/710] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:59.313779 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 347/711] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:31:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:31:59.319105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ddfbcf-0d7c-42ff-8250-e59886be3bbf None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:59.453173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ddfbcf-0d7c-42ff-8250-e59886be3bbf None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:59.458573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd2a26-055b-4a6c-9553-fbd6eacba275 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:59.461288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:59.495380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225281e-7d0b-4b78-8bea-7333e8a9fa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:59.496059 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225281e-7d0b-4b78-8bea-7333e8a9fa21 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:59.496565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225281e-7d0b-4b78-8bea-7333e8a9fa21 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:59.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225281e-7d0b-4b78-8bea-7333e8a9fa21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:59.510668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225281e-7d0b-4b78-8bea-7333e8a9fa21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:59.512164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225281e-7d0b-4b78-8bea-7333e8a9fa21 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:59.540252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-4225281e-7d0b-4b78-8bea-7333e8a9fa21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:31:59.545065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:59.546017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:31:59.547304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:31:59.547304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:31:59.547304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:31:59.547304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 365/712] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:59.552233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225281e-7d0b-4b78-8bea-7333e8a9fa21 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:59.605136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd2a26-055b-4a6c-9553-fbd6eacba275 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:59.640603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0541d7-169d-47af-9584-0a321962257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:59.641244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0541d7-169d-47af-9584-0a321962257b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:59.641534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0541d7-169d-47af-9584-0a321962257b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:59.641816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0541d7-169d-47af-9584-0a321962257b None admin] PATH_INFO: `/v3/services/network` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:59.642409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0541d7-169d-47af-9584-0a321962257b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:59.643799 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0541d7-169d-47af-9584-0a321962257b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-7b0541d7-169d-47af-9584-0a321962257b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:31:59.655083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:31:59.656162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:31:59.657697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 348/713] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:59 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:31:59.662795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0541d7-169d-47af-9584-0a321962257b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:59.747116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225281e-7d0b-4b78-8bea-7333e8a9fa21 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:59.795377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0541d7-169d-47af-9584-0a321962257b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:59.796222 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f5792-cd5c-4fc8-ace9-b7dfc6835ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:59.797329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f5792-cd5c-4fc8-ace9-b7dfc6835ad7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:59.798749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f5792-cd5c-4fc8-ace9-b7dfc6835ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:59.799273 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f5792-cd5c-4fc8-ace9-b7dfc6835ad7 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:59.800177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f5792-cd5c-4fc8-ace9-b7dfc6835ad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:59.800705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f5792-cd5c-4fc8-ace9-b7dfc6835ad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:59.802588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f5792-cd5c-4fc8-ace9-b7dfc6835ad7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:59.804165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:59.819527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 366/714] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:59.820890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149e354-abed-45e4-94f3-2778d53c6662 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:59.821227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149e354-abed-45e4-94f3-2778d53c6662 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:59.821479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149e354-abed-45e4-94f3-2778d53c6662 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:59.827050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 367/715] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:31:59.827050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149e354-abed-45e4-94f3-2778d53c6662 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:31:59.851695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:31:59.853751 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4441c0-c1c6-408e-a362-ab83d905a00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:31:59.854534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4441c0-c1c6-408e-a362-ab83d905a00a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:59.854962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4441c0-c1c6-408e-a362-ab83d905a00a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:59.855380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4441c0-c1c6-408e-a362-ab83d905a00a None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:59.856694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4441c0-c1c6-408e-a362-ab83d905a00a None admin] RBAC: Adding query filter params (name=network) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:31:59.857139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4441c0-c1c6-408e-a362-ab83d905a00a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:31:59.858957 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4441c0-c1c6-408e-a362-ab83d905a00a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:31:59.865598 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 349/716] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:31:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:31:59.867060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bc634-bb03-476a-999e-a7a8aab43124 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:31:59.867503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bc634-bb03-476a-999e-a7a8aab43124 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:31:59.867947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bc634-bb03-476a-999e-a7a8aab43124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:31:59.890200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-dc9bc634-bb03-476a-999e-a7a8aab43124 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:31:59.891586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-dc9bc634-bb03-476a-999e-a7a8aab43124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:31:59.966750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149e354-abed-45e4-94f3-2778d53c6662 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:31:59.970107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 350/717] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:31:59.970857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bc634-bb03-476a-999e-a7a8aab43124 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:00.008241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd59e4a-68df-445c-90d8-76b8478ed31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:00.009640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59e4a-68df-445c-90d8-76b8478ed31c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:00.010075 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59e4a-68df-445c-90d8-76b8478ed31c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:00.010517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59e4a-68df-445c-90d8-76b8478ed31c None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:00.018758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd59e4a-68df-445c-90d8-76b8478ed31c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:00.021232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd59e4a-68df-445c-90d8-76b8478ed31c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-2cd59e4a-68df-445c-90d8-76b8478ed31c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:00.035146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:00.036319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:00.037472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:00.037472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:00.037472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:00.037472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:00.037472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:00.037472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:00.037472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:00.037472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:00.040542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 368/718] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:31:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:00.045665 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd59e4a-68df-445c-90d8-76b8478ed31c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:00.117437 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:00.120253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bc634-bb03-476a-999e-a7a8aab43124 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:00.146311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:00.165187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cb362-9c14-408d-b8ed-92cd3176971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:00.165811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cb362-9c14-408d-b8ed-92cd3176971d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:00.166095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cb362-9c14-408d-b8ed-92cd3176971d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:00.166424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cb362-9c14-408d-b8ed-92cd3176971d None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:00.167069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cb362-9c14-408d-b8ed-92cd3176971d None admin] RBAC: Adding query filter params (type=network) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:00.167375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cb362-9c14-408d-b8ed-92cd3176971d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:00.168766 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cb362-9c14-408d-b8ed-92cd3176971d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:00.172445 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd59e4a-68df-445c-90d8-76b8478ed31c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:00.183138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 351/719] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:31:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:00.206083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a99d48-6308-4609-8f48-a53063196324 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:00.206396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a99d48-6308-4609-8f48-a53063196324 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:00.206718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a99d48-6308-4609-8f48-a53063196324 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:00.209022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 352/720] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:00.218257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64e9bf-3909-4ed9-b604-e12f48ccaeeb None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:00.218257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64e9bf-3909-4ed9-b604-e12f48ccaeeb None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:00.218257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64e9bf-3909-4ed9-b604-e12f48ccaeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:00.219676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac94da3-5b69-4342-bbdc-daf46e955311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:00.220540 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac94da3-5b69-4342-bbdc-daf46e955311 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:00.221073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac94da3-5b69-4342-bbdc-daf46e955311 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:00.221532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac94da3-5b69-4342-bbdc-daf46e955311 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:00.222522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac94da3-5b69-4342-bbdc-daf46e955311 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:00.223115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac94da3-5b69-4342-bbdc-daf46e955311 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:00.225174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac94da3-5b69-4342-bbdc-daf46e955311 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:00.242880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-be64e9bf-3909-4ed9-b604-e12f48ccaeeb None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:00.242880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-be64e9bf-3909-4ed9-b604-e12f48ccaeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:00.255822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 369/721] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:00.261930 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac94da3-5b69-4342-bbdc-daf46e955311 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:00.399473 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 353/722] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:32:00.400375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64e9bf-3909-4ed9-b604-e12f48ccaeeb None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:00.430561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac94da3-5b69-4342-bbdc-daf46e955311 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:00.483584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e24db-5731-4273-aa03-fa7d32ae0e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:00.484457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e24db-5731-4273-aa03-fa7d32ae0e55 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:00.484901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e24db-5731-4273-aa03-fa7d32ae0e55 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:00.485248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e24db-5731-4273-aa03-fa7d32ae0e55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:00.486303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e24db-5731-4273-aa03-fa7d32ae0e55 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=d1b01c21c38949938141eefdc4939385) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:00.486623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e24db-5731-4273-aa03-fa7d32ae0e55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:00.488251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e24db-5731-4273-aa03-fa7d32ae0e55 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:00.515784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 370/723] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:00 2022] GET /identity/v3/role_assignments?user.id=d1b01c21c38949938141eefdc4939385&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=6ba5a5add2174983be37258aad1ee404 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:00.603338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:00.614880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64e9bf-3909-4ed9-b604-e12f48ccaeeb None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:00.665735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eafa2e-8fcd-4c81-914a-698ac9be78a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:00.666391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eafa2e-8fcd-4c81-914a-698ac9be78a1 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:00.666674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eafa2e-8fcd-4c81-914a-698ac9be78a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:00.666953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eafa2e-8fcd-4c81-914a-698ac9be78a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:00.679147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eafa2e-8fcd-4c81-914a-698ac9be78a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-b2eafa2e-8fcd-4c81-914a-698ac9be78a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:00.683922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:00.684924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:00.685806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 354/724] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:32:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:00.692594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e24db-5731-4273-aa03-fa7d32ae0e55 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:00.695562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:00.734669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eafa2e-8fcd-4c81-914a-698ac9be78a1 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:00.885906 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e24db-5731-4273-aa03-fa7d32ae0e55 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:00.905535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eafa2e-8fcd-4c81-914a-698ac9be78a1 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:00.945677 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e057a-7158-4b96-abe1-140964955b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:00.946361 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e057a-7158-4b96-abe1-140964955b43 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:00.946649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e057a-7158-4b96-abe1-140964955b43 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:00.947403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e057a-7158-4b96-abe1-140964955b43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:00.969122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e057a-7158-4b96-abe1-140964955b43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-625e057a-7158-4b96-abe1-140964955b43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:00.976998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:00.978152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:00.979292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:00.980127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ec20f-d1ac-43a6-8a02-efd0e7277979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:00.980127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ec20f-d1ac-43a6-8a02-efd0e7277979 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:00.980314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ec20f-d1ac-43a6-8a02-efd0e7277979 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:00.981313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ec20f-d1ac-43a6-8a02-efd0e7277979 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:00.981313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ec20f-d1ac-43a6-8a02-efd0e7277979 None admin] RBAC: Adding query filter params (name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:00.981496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ec20f-d1ac-43a6-8a02-efd0e7277979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:00.982820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ec20f-d1ac-43a6-8a02-efd0e7277979 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:00.984596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 355/725] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:32:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:00.986882 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e057a-7158-4b96-abe1-140964955b43 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:01.003905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 371/726] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:32:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:01.023453 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:01.024320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ec20f-d1ac-43a6-8a02-efd0e7277979 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:01.116858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:01.161046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ec20f-d1ac-43a6-8a02-efd0e7277979 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:01.204018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dd84b-bdf8-46d2-8e77-c62d8772bf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:01.204751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dd84b-bdf8-46d2-8e77-c62d8772bf32 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:01.205096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dd84b-bdf8-46d2-8e77-c62d8772bf32 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:01.205435 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dd84b-bdf8-46d2-8e77-c62d8772bf32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:01.218224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dd84b-bdf8-46d2-8e77-c62d8772bf32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:01.220987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dd84b-bdf8-46d2-8e77-c62d8772bf32 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:01.227440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e057a-7158-4b96-abe1-140964955b43 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-445dd84b-bdf8-46d2-8e77-c62d8772bf32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:01.246298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:01.247564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:01.248826 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:01.248826 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:01.248826 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:01.267971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 372/727] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:01.267971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dd84b-bdf8-46d2-8e77-c62d8772bf32 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:01.319400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07295e25-76c0-401f-b905-dee98713b7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:01.332046 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07295e25-76c0-401f-b905-dee98713b7e7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:01.332046 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07295e25-76c0-401f-b905-dee98713b7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:01.332046 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07295e25-76c0-401f-b905-dee98713b7e7 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:01.332046 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07295e25-76c0-401f-b905-dee98713b7e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:01.332046 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07295e25-76c0-401f-b905-dee98713b7e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:01.332046 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07295e25-76c0-401f-b905-dee98713b7e7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:01.345075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 356/728] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:32:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:01.348824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f30d2-036b-43cd-9fdd-a192a5dc4312 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:01.349386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f30d2-036b-43cd-9fdd-a192a5dc4312 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:01.350458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f30d2-036b-43cd-9fdd-a192a5dc4312 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:01.364579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 357/729] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:01 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:01.365561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f30d2-036b-43cd-9fdd-a192a5dc4312 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:01.400293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dd84b-bdf8-46d2-8e77-c62d8772bf32 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:01.433992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:01.439653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e410e-707c-4e2c-aa1a-aebf4160bba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:01.440376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e410e-707c-4e2c-aa1a-aebf4160bba5 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:01.440718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e410e-707c-4e2c-aa1a-aebf4160bba5 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:01.440999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e410e-707c-4e2c-aa1a-aebf4160bba5 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:01.441692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e410e-707c-4e2c-aa1a-aebf4160bba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:01.442055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e410e-707c-4e2c-aa1a-aebf4160bba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:01.443437 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e410e-707c-4e2c-aa1a-aebf4160bba5 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:01.460078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 373/730] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:01.461583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f671-3fe0-4bd4-b033-04a663c7bf10 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:01.461883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f671-3fe0-4bd4-b033-04a663c7bf10 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:01.462164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f671-3fe0-4bd4-b033-04a663c7bf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:01.482952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0152f671-3fe0-4bd4-b033-04a663c7bf10 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:01.484126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0152f671-3fe0-4bd4-b033-04a663c7bf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:01.540283 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:01.555982 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 374/731] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:32:01.557051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152f671-3fe0-4bd4-b033-04a663c7bf10 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:01.630340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f30d2-036b-43cd-9fdd-a192a5dc4312 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:01.761272 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5d497-38eb-4c31-9e34-e2f74f53aa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:01.762285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5d497-38eb-4c31-9e34-e2f74f53aa58 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:01.762774 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5d497-38eb-4c31-9e34-e2f74f53aa58 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:01.763250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5d497-38eb-4c31-9e34-e2f74f53aa58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:01.784344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5d497-38eb-4c31-9e34-e2f74f53aa58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:01.787787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5d497-38eb-4c31-9e34-e2f74f53aa58 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-fbf5d497-38eb-4c31-9e34-e2f74f53aa58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:01.823141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:01.824189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:01.825119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:01.825119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:01.825119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:01.827321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 358/732] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:01.837087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5d497-38eb-4c31-9e34-e2f74f53aa58 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:01.879050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152f671-3fe0-4bd4-b033-04a663c7bf10 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:01.888475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:01.941775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72195c95-dd71-4544-afaa-1a45684a6615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:01.942607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72195c95-dd71-4544-afaa-1a45684a6615 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:01.943040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72195c95-dd71-4544-afaa-1a45684a6615 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:01.943322 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72195c95-dd71-4544-afaa-1a45684a6615 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:01.953068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72195c95-dd71-4544-afaa-1a45684a6615 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:01.955241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72195c95-dd71-4544-afaa-1a45684a6615 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-72195c95-dd71-4544-afaa-1a45684a6615 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:01.971560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:01.972995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 375/733] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Aug 26 15:32:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:01.974621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72195c95-dd71-4544-afaa-1a45684a6615 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:02.004799 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5d497-38eb-4c31-9e34-e2f74f53aa58 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:02.043793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:02.052897 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b5247-40ff-4a17-b922-b86389a1c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:02.053551 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5247-40ff-4a17-b922-b86389a1c299 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:02.059386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5247-40ff-4a17-b922-b86389a1c299 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:02.059386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5247-40ff-4a17-b922-b86389a1c299 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:02.059386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b5247-40ff-4a17-b922-b86389a1c299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:02.059386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b5247-40ff-4a17-b922-b86389a1c299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:02.059386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b5247-40ff-4a17-b922-b86389a1c299 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:02.074174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 359/734] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:02.075231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b5247-40ff-4a17-b922-b86389a1c299 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:02.107066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72195c95-dd71-4544-afaa-1a45684a6615 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:02.149551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d42148-3cd1-46af-9675-c138e6546b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:02.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42148-3cd1-46af-9675-c138e6546b13 None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:02.150973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42148-3cd1-46af-9675-c138e6546b13 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:02.151524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42148-3cd1-46af-9675-c138e6546b13 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:02.152222 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d42148-3cd1-46af-9675-c138e6546b13 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:02.154516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d42148-3cd1-46af-9675-c138e6546b13 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:02.186031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b5247-40ff-4a17-b922-b86389a1c299 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:02.223308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb7180-9f96-4501-b83e-8b134219e346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:02.223952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb7180-9f96-4501-b83e-8b134219e346 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:02.224227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb7180-9f96-4501-b83e-8b134219e346 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:02.224503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb7180-9f96-4501-b83e-8b134219e346 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:02.225329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb7180-9f96-4501-b83e-8b134219e346 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:02.225533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb7180-9f96-4501-b83e-8b134219e346 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:02.227095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb7180-9f96-4501-b83e-8b134219e346 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:02.249876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 360/735] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Aug 26 15:32:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:02.249876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb7180-9f96-4501-b83e-8b134219e346 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:02.262383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 376/736] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:32:01 2022] POST /identity/v3/services => generated 251 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:02.264160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d42148-3cd1-46af-9675-c138e6546b13 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:02.315693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:02.346729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:02.429803 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb7180-9f96-4501-b83e-8b134219e346 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:02.448668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d42148-3cd1-46af-9675-c138e6546b13 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:02.475534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa3e5a-c495-49fa-99d9-0381b21eea84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:02.476227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa3e5a-c495-49fa-99d9-0381b21eea84 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:02.476513 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa3e5a-c495-49fa-99d9-0381b21eea84 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:02.476792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa3e5a-c495-49fa-99d9-0381b21eea84 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:02.484892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa3e5a-c495-49fa-99d9-0381b21eea84 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:02.486339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa3e5a-c495-49fa-99d9-0381b21eea84 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-33fa3e5a-c495-49fa-99d9-0381b21eea84 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:02.499173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:02.500216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:02.501576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:02.501576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:02.501576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:02.501576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:02.501576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:02.501576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:02.501576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:02.501576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:02.501576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:32:02.501576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:02.501576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 361/737] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:32:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:02.502174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e4d0-e9dd-44ef-9cb9-bf4a2b60278f None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:02.502174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e4d0-e9dd-44ef-9cb9-bf4a2b60278f None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:02.502510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e4d0-e9dd-44ef-9cb9-bf4a2b60278f None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:02.503777 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 362/738] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:02.510333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a67e4-0447-4b06-8e42-79f6ff074073 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:02.510619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a67e4-0447-4b06-8e42-79f6ff074073 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:02.511139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a67e4-0447-4b06-8e42-79f6ff074073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:02.526362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae812c0a-17f2-4433-831d-37e3e887796d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:02.527347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae812c0a-17f2-4433-831d-37e3e887796d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:02.527835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae812c0a-17f2-4433-831d-37e3e887796d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:02.528359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae812c0a-17f2-4433-831d-37e3e887796d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:02.535514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-de6a67e4-0447-4b06-8e42-79f6ff074073 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:02.536771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-de6a67e4-0447-4b06-8e42-79f6ff074073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:02.612732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae812c0a-17f2-4433-831d-37e3e887796d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:02.615454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae812c0a-17f2-4433-831d-37e3e887796d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:02.627166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 363/739] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:32:02.628514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a67e4-0447-4b06-8e42-79f6ff074073 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:02.642586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:02.676497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:02.758516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 377/740] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:32:02.764066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae812c0a-17f2-4433-831d-37e3e887796d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:02.805853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a67e4-0447-4b06-8e42-79f6ff074073 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:02.855635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafd516-8d57-4e44-bb08-1968f4757b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:02.856581 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafd516-8d57-4e44-bb08-1968f4757b46 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:02.857255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafd516-8d57-4e44-bb08-1968f4757b46 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:02.857549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafd516-8d57-4e44-bb08-1968f4757b46 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:02.858636 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafd516-8d57-4e44-bb08-1968f4757b46 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:02.858945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafd516-8d57-4e44-bb08-1968f4757b46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:02.861330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafd516-8d57-4e44-bb08-1968f4757b46 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:02.886081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 364/741] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Aug 26 15:32:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:02.887352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967dae8-a6e2-4028-ab56-b4b9b9af32fd None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:02.887643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967dae8-a6e2-4028-ab56-b4b9b9af32fd None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:02.887968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967dae8-a6e2-4028-ab56-b4b9b9af32fd None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:02.889417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 365/742] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:02.892417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967dae8-a6e2-4028-ab56-b4b9b9af32fd None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:02.916739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae812c0a-17f2-4433-831d-37e3e887796d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:02.945815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:02.958206 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0e9c6-54be-4116-9fed-1f2bcc9f6a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:02.960080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0e9c6-54be-4116-9fed-1f2bcc9f6a1d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:02.960080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0e9c6-54be-4116-9fed-1f2bcc9f6a1d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:02.960080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0e9c6-54be-4116-9fed-1f2bcc9f6a1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:02.971230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0e9c6-54be-4116-9fed-1f2bcc9f6a1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:02.973302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0e9c6-54be-4116-9fed-1f2bcc9f6a1d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-1ac0e9c6-54be-4116-9fed-1f2bcc9f6a1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:02.987727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:02.989477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:02.990907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:02.990907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:02.990907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:02.990907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 378/743] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:02.990907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69688f-947f-4cd8-98e2-d0fbff03ca9a None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:02.991205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69688f-947f-4cd8-98e2-d0fbff03ca9a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:02.991638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69688f-947f-4cd8-98e2-d0fbff03ca9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:03.010129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1a69688f-947f-4cd8-98e2-d0fbff03ca9a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:03.011597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1a69688f-947f-4cd8-98e2-d0fbff03ca9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:03.012238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:03.025048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967dae8-a6e2-4028-ab56-b4b9b9af32fd None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:03.067315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f535d-fbff-4498-89c7-b95f9e1da96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:03.067954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f535d-fbff-4498-89c7-b95f9e1da96c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:03.068240 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f535d-fbff-4498-89c7-b95f9e1da96c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:03.068512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f535d-fbff-4498-89c7-b95f9e1da96c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:03.091941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 379/744] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:32:03.091941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69688f-947f-4cd8-98e2-d0fbff03ca9a None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:03.148340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f535d-fbff-4498-89c7-b95f9e1da96c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:03.150039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f535d-fbff-4498-89c7-b95f9e1da96c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:03.209331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69688f-947f-4cd8-98e2-d0fbff03ca9a None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:03.213673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:03.253824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c408f-2580-4f08-b82e-ec79e01c65b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:03.254803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c408f-2580-4f08-b82e-ec79e01c65b7 None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:03.255155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c408f-2580-4f08-b82e-ec79e01c65b7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:03.255508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c408f-2580-4f08-b82e-ec79e01c65b7 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:03.256446 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:03.257294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c408f-2580-4f08-b82e-ec79e01c65b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:03.259222 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c408f-2580-4f08-b82e-ec79e01c65b7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:03.265459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 366/745] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:32:03.266551 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f535d-fbff-4498-89c7-b95f9e1da96c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:03.344321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 380/746] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:32:03 2022] POST /identity/v3/services => generated 240 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:03.347114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c408f-2580-4f08-b82e-ec79e01c65b7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:03.415547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f535d-fbff-4498-89c7-b95f9e1da96c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:03.481020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3a938-18a4-4dbd-bd11-11d035afc0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:03.481823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3a938-18a4-4dbd-bd11-11d035afc0e6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:03.482110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3a938-18a4-4dbd-bd11-11d035afc0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:03.482392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3a938-18a4-4dbd-bd11-11d035afc0e6 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:03.483454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3a938-18a4-4dbd-bd11-11d035afc0e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:03.483772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3a938-18a4-4dbd-bd11-11d035afc0e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:03.485704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3a938-18a4-4dbd-bd11-11d035afc0e6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:03.502676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c408f-2580-4f08-b82e-ec79e01c65b7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:03.504346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 367/747] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:03.506587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3a938-18a4-4dbd-bd11-11d035afc0e6 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:03.558664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997dd69-53ba-49d5-8283-d26b8a1e3a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:03.559462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997dd69-53ba-49d5-8283-d26b8a1e3a6e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:03.559783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997dd69-53ba-49d5-8283-d26b8a1e3a6e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:03.560094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997dd69-53ba-49d5-8283-d26b8a1e3a6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:03.568745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:03.573762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997dd69-53ba-49d5-8283-d26b8a1e3a6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:03.574732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997dd69-53ba-49d5-8283-d26b8a1e3a6e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-6997dd69-53ba-49d5-8283-d26b8a1e3a6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:03.595479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:03.597959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:03.599562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:03.599562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:03.599562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:03.599562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 381/748] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:03.599562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997dd69-53ba-49d5-8283-d26b8a1e3a6e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:03.621333 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:03.649495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3a938-18a4-4dbd-bd11-11d035afc0e6 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:03.696401 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef04209-9265-464f-b630-f1d9dda7c2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:03.697942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef04209-9265-464f-b630-f1d9dda7c2a7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:03.698431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef04209-9265-464f-b630-f1d9dda7c2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:03.698910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef04209-9265-464f-b630-f1d9dda7c2a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:03.710070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997dd69-53ba-49d5-8283-d26b8a1e3a6e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:03.711645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef04209-9265-464f-b630-f1d9dda7c2a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-8ef04209-9265-464f-b630-f1d9dda7c2a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:03.716733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:03.717698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:03.718570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:03.721417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 368/749] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:32:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:03.722635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef04209-9265-464f-b630-f1d9dda7c2a7 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:03.757006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77041d2c-9e2a-4229-8125-fd6f64987668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:03.757918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77041d2c-9e2a-4229-8125-fd6f64987668 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:03.758207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77041d2c-9e2a-4229-8125-fd6f64987668 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:03.758486 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77041d2c-9e2a-4229-8125-fd6f64987668 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:03.775000 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77041d2c-9e2a-4229-8125-fd6f64987668 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:03.777034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77041d2c-9e2a-4229-8125-fd6f64987668 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-77041d2c-9e2a-4229-8125-fd6f64987668 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:03.794082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:03.795037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:03.795963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:03.795963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:03.795963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:03.795963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:03.795963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:03.795963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:03.795963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:03.795963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:03.796889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 382/750] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:03.796959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77041d2c-9e2a-4229-8125-fd6f64987668 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:03.847206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef04209-9265-464f-b630-f1d9dda7c2a7 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:03.860581 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:03.890623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7440d2-5316-447a-b992-bfca9822c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:03.891823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7440d2-5316-447a-b992-bfca9822c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:03.892151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7440d2-5316-447a-b992-bfca9822c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:03.892599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7440d2-5316-447a-b992-bfca9822c7f3 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:03.893378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7440d2-5316-447a-b992-bfca9822c7f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:03.893691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7440d2-5316-447a-b992-bfca9822c7f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:03.897003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7440d2-5316-447a-b992-bfca9822c7f3 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:03.906667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:03.913204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 369/751] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:03.914906 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7440d2-5316-447a-b992-bfca9822c7f3 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:03.934371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77041d2c-9e2a-4229-8125-fd6f64987668 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:03.976291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c9a6-7975-46cc-a4e8-7e81dbf23c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:03.976980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c9a6-7975-46cc-a4e8-7e81dbf23c1b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:03.977580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c9a6-7975-46cc-a4e8-7e81dbf23c1b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:03.977700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c9a6-7975-46cc-a4e8-7e81dbf23c1b None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:03.978448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36c9a6-7975-46cc-a4e8-7e81dbf23c1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:03.978903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36c9a6-7975-46cc-a4e8-7e81dbf23c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:03.980889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36c9a6-7975-46cc-a4e8-7e81dbf23c1b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:03.991085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 383/752] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:32:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:03.997156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c9a6-7975-46cc-a4e8-7e81dbf23c1b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:04.054047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7440d2-5316-447a-b992-bfca9822c7f3 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:04.095251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf76c5b-4dcb-4c33-810c-0f9346e3de13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:04.096198 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76c5b-4dcb-4c33-810c-0f9346e3de13 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:04.096589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76c5b-4dcb-4c33-810c-0f9346e3de13 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:04.096981 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76c5b-4dcb-4c33-810c-0f9346e3de13 None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:04.098400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf76c5b-4dcb-4c33-810c-0f9346e3de13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:04.098814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf76c5b-4dcb-4c33-810c-0f9346e3de13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:04.100410 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf76c5b-4dcb-4c33-810c-0f9346e3de13 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:04.120270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 370/753] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:04.121412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf76c5b-4dcb-4c33-810c-0f9346e3de13 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:04.130050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c9a6-7975-46cc-a4e8-7e81dbf23c1b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:04.157761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:04.178121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f7921-f68e-4e5a-9420-e63951adf5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:04.178910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f7921-f68e-4e5a-9420-e63951adf5b1 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:04.179300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f7921-f68e-4e5a-9420-e63951adf5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:04.179984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f7921-f68e-4e5a-9420-e63951adf5b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:04.188536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f7921-f68e-4e5a-9420-e63951adf5b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:04.190665 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f7921-f68e-4e5a-9420-e63951adf5b1 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:04.193144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-709f7921-f68e-4e5a-9420-e63951adf5b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:04.205527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:04.208909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:04.210040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:04.210040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:04.210040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:04.210040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:04.210040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:04.210040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:04.210040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:04.210040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:04.210040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 384/754] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:04.211230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f7921-f68e-4e5a-9420-e63951adf5b1 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:04.258156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf76c5b-4dcb-4c33-810c-0f9346e3de13 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:04.303064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ae45e-6763-4e1e-bda2-4cd9ca6e00e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:04.303905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ae45e-6763-4e1e-bda2-4cd9ca6e00e0 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:04.304279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ae45e-6763-4e1e-bda2-4cd9ca6e00e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:04.304702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ae45e-6763-4e1e-bda2-4cd9ca6e00e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:04.316131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ae45e-6763-4e1e-bda2-4cd9ca6e00e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:04.317878 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ae45e-6763-4e1e-bda2-4cd9ca6e00e0 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-619ae45e-6763-4e1e-bda2-4cd9ca6e00e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:04.334681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:04.335717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:04.336695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:04.336695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:04.336695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:04.336695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 371/755] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:04.336695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ae45e-6763-4e1e-bda2-4cd9ca6e00e0 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:04.337308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f7921-f68e-4e5a-9420-e63951adf5b1 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:04.372854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc662383-4423-4b93-8963-fa602e779c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:04.373459 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662383-4423-4b93-8963-fa602e779c18 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:04.373742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662383-4423-4b93-8963-fa602e779c18 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:04.374011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662383-4423-4b93-8963-fa602e779c18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:04.374737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc662383-4423-4b93-8963-fa602e779c18 None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=b24a6d0c1dbe49a18b3b8b6732a938be) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:04.375046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc662383-4423-4b93-8963-fa602e779c18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:04.376520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc662383-4423-4b93-8963-fa602e779c18 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:04.388014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 385/756] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:04 2022] GET /identity/v3/role_assignments?user.id=b24a6d0c1dbe49a18b3b8b6732a938be&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:04.389020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc662383-4423-4b93-8963-fa602e779c18 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:04.441821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:04.463892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ae45e-6763-4e1e-bda2-4cd9ca6e00e0 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:04.493141 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:04.502903 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec0611-c990-42c3-a317-a0a4585c7a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:04.504654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec0611-c990-42c3-a317-a0a4585c7a9d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:04.504654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec0611-c990-42c3-a317-a0a4585c7a9d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:04.504654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec0611-c990-42c3-a317-a0a4585c7a9d None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:04.505381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec0611-c990-42c3-a317-a0a4585c7a9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:04.505683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec0611-c990-42c3-a317-a0a4585c7a9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:04.507176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec0611-c990-42c3-a317-a0a4585c7a9d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:04.521735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 372/757] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:04.522960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1224-ffdd-4c34-80cc-86714076fdb6 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:04.523246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1224-ffdd-4c34-80cc-86714076fdb6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:04.523541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1224-ffdd-4c34-80cc-86714076fdb6 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:04.529396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 373/758] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:04.529396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f1224-ffdd-4c34-80cc-86714076fdb6 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:04.542748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc662383-4423-4b93-8963-fa602e779c18 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:04.585432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e9068-5c82-4b78-8166-5680b274b366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:04.586182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e9068-5c82-4b78-8166-5680b274b366 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:04.586599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e9068-5c82-4b78-8166-5680b274b366 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:04.586886 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e9068-5c82-4b78-8166-5680b274b366 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:04.587537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e9068-5c82-4b78-8166-5680b274b366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:04.587843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e9068-5c82-4b78-8166-5680b274b366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:04.589408 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e9068-5c82-4b78-8166-5680b274b366 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:04.605277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 386/759] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:04.606918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c37a55-b209-46e4-aeeb-eacf286099ab None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:04.607213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c37a55-b209-46e4-aeeb-eacf286099ab None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:04.607496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c37a55-b209-46e4-aeeb-eacf286099ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:04.630700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-06c37a55-b209-46e4-aeeb-eacf286099ab None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:04.632208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-06c37a55-b209-46e4-aeeb-eacf286099ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:04.654448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f1224-ffdd-4c34-80cc-86714076fdb6 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:04.703929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3497ac-e8a9-4df5-87c1-028d77d6bbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:04.704716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3497ac-e8a9-4df5-87c1-028d77d6bbbe None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:04.704942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3497ac-e8a9-4df5-87c1-028d77d6bbbe None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:04.705222 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3497ac-e8a9-4df5-87c1-028d77d6bbbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:04.705970 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3497ac-e8a9-4df5-87c1-028d77d6bbbe None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=1fae47d6b7a7472a9d94636bb126f802) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:04.707089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3497ac-e8a9-4df5-87c1-028d77d6bbbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:04.709291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3497ac-e8a9-4df5-87c1-028d77d6bbbe None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:04.713901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 387/760] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:04.714999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c37a55-b209-46e4-aeeb-eacf286099ab None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:04.722941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 374/761] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:04 2022] GET /identity/v3/role_assignments?user.id=1fae47d6b7a7472a9d94636bb126f802&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=6ba5a5add2174983be37258aad1ee404 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:04.808299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:04.872371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c37a55-b209-46e4-aeeb-eacf286099ab None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:04.917704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21ae29-5862-4857-8ba1-f225fc0e0a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:04.918343 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21ae29-5862-4857-8ba1-f225fc0e0a61 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:04.918631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21ae29-5862-4857-8ba1-f225fc0e0a61 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:04.918909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21ae29-5862-4857-8ba1-f225fc0e0a61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:04.927030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21ae29-5862-4857-8ba1-f225fc0e0a61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:04.928548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21ae29-5862-4857-8ba1-f225fc0e0a61 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-ae21ae29-5862-4857-8ba1-f225fc0e0a61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:04.940550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:04.941559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:04.942976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:04.942976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:04.942976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:04.942976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:04.942976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:04.942976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:04.942976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:04.942976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:04.942976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:32:04.942976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:04.942976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 388/762] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Aug 26 15:32:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:04.947407 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3497ac-e8a9-4df5-87c1-028d77d6bbbe None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:04.957997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:05.054367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3497ac-e8a9-4df5-87c1-028d77d6bbbe None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:05.091043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b0c7f-ce1b-4dd2-8118-83147b2f9100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:05.091580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b0c7f-ce1b-4dd2-8118-83147b2f9100 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:05.091889 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b0c7f-ce1b-4dd2-8118-83147b2f9100 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:05.092170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b0c7f-ce1b-4dd2-8118-83147b2f9100 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:05.092914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b0c7f-ce1b-4dd2-8118-83147b2f9100 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:05.093237 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b0c7f-ce1b-4dd2-8118-83147b2f9100 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:05.094672 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b0c7f-ce1b-4dd2-8118-83147b2f9100 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:05.114565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 375/763] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Aug 26 15:32:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:05.120960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21ae29-5862-4857-8ba1-f225fc0e0a61 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:05.252296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21ae29-5862-4857-8ba1-f225fc0e0a61 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:05.267103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b0c7f-ce1b-4dd2-8118-83147b2f9100 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:05.279433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:05.308883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57277d-2565-4b5c-b47d-2a13d905e8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:05.309512 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57277d-2565-4b5c-b47d-2a13d905e8b8 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:05.309794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57277d-2565-4b5c-b47d-2a13d905e8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:05.310066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57277d-2565-4b5c-b47d-2a13d905e8b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:05.330112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57277d-2565-4b5c-b47d-2a13d905e8b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:05.331652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57277d-2565-4b5c-b47d-2a13d905e8b8 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-9a57277d-2565-4b5c-b47d-2a13d905e8b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:05.346404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:05.347984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:05.349530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:05.349530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:05.349530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:05.349530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 389/764] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:05.353505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:05.354094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57277d-2565-4b5c-b47d-2a13d905e8b8 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:05.428102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b0c7f-ce1b-4dd2-8118-83147b2f9100 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:05.496146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38d4ae-3eec-40aa-8ce1-f60251f1d2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:05.498445 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d4ae-3eec-40aa-8ce1-f60251f1d2ef None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:05.498731 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d4ae-3eec-40aa-8ce1-f60251f1d2ef None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:05.499018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d4ae-3eec-40aa-8ce1-f60251f1d2ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:05.499742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38d4ae-3eec-40aa-8ce1-f60251f1d2ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:05.501158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38d4ae-3eec-40aa-8ce1-f60251f1d2ef None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-6e38d4ae-3eec-40aa-8ce1-f60251f1d2ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:05.513894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:05.514971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:32:05.515913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:05.516772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57277d-2565-4b5c-b47d-2a13d905e8b8 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:05.519009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 376/765] 10.222.0.98 () {56 vars in 1129 bytes} [Fri Aug 26 15:32:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:05.520667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaec329-d970-477d-8a06-c3b771a39ac2 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:05.521183 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaec329-d970-477d-8a06-c3b771a39ac2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:05.521666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaec329-d970-477d-8a06-c3b771a39ac2 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:05.527649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 377/766] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:05.528764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaec329-d970-477d-8a06-c3b771a39ac2 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:05.556418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b8ac1-bbc1-48d6-85e1-d741bef70534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:05.557711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b8ac1-bbc1-48d6-85e1-d741bef70534 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:05.558466 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b8ac1-bbc1-48d6-85e1-d741bef70534 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:05.559500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b8ac1-bbc1-48d6-85e1-d741bef70534 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:05.560702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b8ac1-bbc1-48d6-85e1-d741bef70534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:05.561488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b8ac1-bbc1-48d6-85e1-d741bef70534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:05.563848 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b8ac1-bbc1-48d6-85e1-d741bef70534 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:05.596169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 390/767] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:05.596169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85ea0-9521-409d-a7dd-a74c202fbc14 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:05.596169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85ea0-9521-409d-a7dd-a74c202fbc14 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:05.596169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85ea0-9521-409d-a7dd-a74c202fbc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:05.616078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1dc85ea0-9521-409d-a7dd-a74c202fbc14 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:05.617880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1dc85ea0-9521-409d-a7dd-a74c202fbc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:05.638829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:05.663952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaec329-d970-477d-8a06-c3b771a39ac2 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:05.692090 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 391/768] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:05.693099 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc85ea0-9521-409d-a7dd-a74c202fbc14 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:05.704227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:05.720573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145de0d-f221-4d29-bb07-6a1b436c1a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:05.732100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145de0d-f221-4d29-bb07-6a1b436c1a48 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:05.732100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145de0d-f221-4d29-bb07-6a1b436c1a48 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:05.732100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145de0d-f221-4d29-bb07-6a1b436c1a48 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:05.732100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145de0d-f221-4d29-bb07-6a1b436c1a48 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:05.732100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145de0d-f221-4d29-bb07-6a1b436c1a48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:05.732100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145de0d-f221-4d29-bb07-6a1b436c1a48 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:05.744473 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 378/769] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Aug 26 15:32:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:05.752606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145de0d-f221-4d29-bb07-6a1b436c1a48 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:05.875281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc85ea0-9521-409d-a7dd-a74c202fbc14 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:05.922149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa6a2f-4032-4cba-a4f7-aae2e04896ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:05.923154 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa6a2f-4032-4cba-a4f7-aae2e04896ba None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:05.923660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa6a2f-4032-4cba-a4f7-aae2e04896ba None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:05.924184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa6a2f-4032-4cba-a4f7-aae2e04896ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:05.938760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa6a2f-4032-4cba-a4f7-aae2e04896ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:05.941803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa6a2f-4032-4cba-a4f7-aae2e04896ba None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-d6aa6a2f-4032-4cba-a4f7-aae2e04896ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:05.963768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:05.964792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:05.965744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:05.965744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:05.965744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:05.965744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:05.965744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:05.965744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:05.965744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:05.965744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:05.966192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145de0d-f221-4d29-bb07-6a1b436c1a48 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:05.967930 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 392/770] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:05.968905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa6a2f-4032-4cba-a4f7-aae2e04896ba None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:06.006999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028cff9-8996-4979-a7c6-7a4d71f9509d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:06.007817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028cff9-8996-4979-a7c6-7a4d71f9509d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.008320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028cff9-8996-4979-a7c6-7a4d71f9509d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.008678 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028cff9-8996-4979-a7c6-7a4d71f9509d None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.009754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028cff9-8996-4979-a7c6-7a4d71f9509d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:06.010135 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028cff9-8996-4979-a7c6-7a4d71f9509d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:06.012079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028cff9-8996-4979-a7c6-7a4d71f9509d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:06.023843 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 379/771] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Aug 26 15:32:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:06.024034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:06.024117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028cff9-8996-4979-a7c6-7a4d71f9509d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:06.094090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:06.108240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa6a2f-4032-4cba-a4f7-aae2e04896ba None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:06.150888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028cff9-8996-4979-a7c6-7a4d71f9509d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:06.154103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd09cf-da60-4305-9b64-e6fec69e719a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:06.154751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd09cf-da60-4305-9b64-e6fec69e719a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.155153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd09cf-da60-4305-9b64-e6fec69e719a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.155544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd09cf-da60-4305-9b64-e6fec69e719a None admin] PATH_INFO: `/v3/services/network` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.156342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd09cf-da60-4305-9b64-e6fec69e719a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:06.157859 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd09cf-da60-4305-9b64-e6fec69e719a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-bcfd09cf-da60-4305-9b64-e6fec69e719a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:06.168582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:06.169673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:32:06.170528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:06.175399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 393/772] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:05 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:06.176422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd09cf-da60-4305-9b64-e6fec69e719a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:06.192521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeca9da-0243-417f-a86e-cacf1cbfe7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:06.193423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeca9da-0243-417f-a86e-cacf1cbfe7bd None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.193914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeca9da-0243-417f-a86e-cacf1cbfe7bd None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.194407 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeca9da-0243-417f-a86e-cacf1cbfe7bd None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.195326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeca9da-0243-417f-a86e-cacf1cbfe7bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:06.195852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeca9da-0243-417f-a86e-cacf1cbfe7bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:06.197510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeca9da-0243-417f-a86e-cacf1cbfe7bd None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:06.216942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 380/773] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:06.218171 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeca9da-0243-417f-a86e-cacf1cbfe7bd None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:06.266919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:06.293038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd09cf-da60-4305-9b64-e6fec69e719a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:06.325246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeca9da-0243-417f-a86e-cacf1cbfe7bd None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:06.343503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:06.344240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f6558-251c-484e-aad1-7b9ff053c105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:06.345940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f6558-251c-484e-aad1-7b9ff053c105 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.346395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f6558-251c-484e-aad1-7b9ff053c105 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.347457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f6558-251c-484e-aad1-7b9ff053c105 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.348631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f6558-251c-484e-aad1-7b9ff053c105 None admin] RBAC: Adding query filter params (interface=public, service_id=dac6b4e5a1084cc2bdf675455151acb3, region_id=RegionOne) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:06.349124 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f6558-251c-484e-aad1-7b9ff053c105 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:06.351491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f6558-251c-484e-aad1-7b9ff053c105 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:06.365333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78138f2-b5ce-43fe-aa11-4dd16d3397a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:06.366306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78138f2-b5ce-43fe-aa11-4dd16d3397a5 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.366816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78138f2-b5ce-43fe-aa11-4dd16d3397a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.367312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78138f2-b5ce-43fe-aa11-4dd16d3397a5 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.368431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78138f2-b5ce-43fe-aa11-4dd16d3397a5 None admin] RBAC: Adding query filter params (name=network) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:06.368980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78138f2-b5ce-43fe-aa11-4dd16d3397a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:06.370682 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78138f2-b5ce-43fe-aa11-4dd16d3397a5 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:06.372233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 394/774] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Aug 26 15:32:06 2022] GET /identity/v3/endpoints?service_id=dac6b4e5a1084cc2bdf675455151acb3&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:06.373632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f6558-251c-484e-aad1-7b9ff053c105 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:06.377089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 381/775] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:32:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:06.378252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78138f2-b5ce-43fe-aa11-4dd16d3397a5 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:06.486953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78138f2-b5ce-43fe-aa11-4dd16d3397a5 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:06.506678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f6558-251c-484e-aad1-7b9ff053c105 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:06.524832 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de0836-539d-4fca-8663-f8d98ed71f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:06.525853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de0836-539d-4fca-8663-f8d98ed71f5b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.526342 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de0836-539d-4fca-8663-f8d98ed71f5b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.526825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de0836-539d-4fca-8663-f8d98ed71f5b None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.527656 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de0836-539d-4fca-8663-f8d98ed71f5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:06.529300 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de0836-539d-4fca-8663-f8d98ed71f5b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:06.535139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 382/776] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Aug 26 15:32:06 2022] GET /identity/v3/services => generated 772 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:06.544821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de0836-539d-4fca-8663-f8d98ed71f5b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:06.551321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3f524-ebca-440f-84a0-93e8fb2bfa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:06.552030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f524-ebca-440f-84a0-93e8fb2bfa54 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.552317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f524-ebca-440f-84a0-93e8fb2bfa54 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.552594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f524-ebca-440f-84a0-93e8fb2bfa54 None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/d1b01c21c38949938141eefdc4939385/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.555753 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3f524-ebca-440f-84a0-93e8fb2bfa54 None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=d1b01c21c38949938141eefdc4939385, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:06.562867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3f524-ebca-440f-84a0-93e8fb2bfa54 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:06.620484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:06.667400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 395/777] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:32:06 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/d1b01c21c38949938141eefdc4939385/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:32:06.670473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c787e1-d215-46ea-91e1-32c2b363c0e7 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.670473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c787e1-d215-46ea-91e1-32c2b363c0e7 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.670473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c787e1-d215-46ea-91e1-32c2b363c0e7 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.670473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 396/778] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:06.677682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329336b7-bd46-4ddc-bccd-0530cf97d40e None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.678005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329336b7-bd46-4ddc-bccd-0530cf97d40e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.678230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329336b7-bd46-4ddc-bccd-0530cf97d40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.701572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:06.713627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de0836-539d-4fca-8663-f8d98ed71f5b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:06.717196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-329336b7-bd46-4ddc-bccd-0530cf97d40e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:06.718433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-329336b7-bd46-4ddc-bccd-0530cf97d40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:06.759406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e111cfa-892a-4eaf-bdb1-750266e0e6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:06.760093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111cfa-892a-4eaf-bdb1-750266e0e6af None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.760421 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111cfa-892a-4eaf-bdb1-750266e0e6af None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.760689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111cfa-892a-4eaf-bdb1-750266e0e6af None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.761302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e111cfa-892a-4eaf-bdb1-750266e0e6af None admin] RBAC: Adding query filter params (type=network) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:06.761607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e111cfa-892a-4eaf-bdb1-750266e0e6af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:06.763042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e111cfa-892a-4eaf-bdb1-750266e0e6af None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:06.769892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 383/779] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:32:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:06.784694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e111cfa-892a-4eaf-bdb1-750266e0e6af None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:06.823535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 397/780] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:06.855369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2243a9-8b9c-4ff5-b1e6-ad8334e634b0 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.855660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2243a9-8b9c-4ff5-b1e6-ad8334e634b0 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.855970 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2243a9-8b9c-4ff5-b1e6-ad8334e634b0 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.857667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 398/781] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:06.865660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f926870-fbf2-45f1-ae35-4644a32d56b0 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.865955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f926870-fbf2-45f1-ae35-4644a32d56b0 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.866228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f926870-fbf2-45f1-ae35-4644a32d56b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.887356 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9f926870-fbf2-45f1-ae35-4644a32d56b0 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:06.887700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9f926870-fbf2-45f1-ae35-4644a32d56b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:06.926170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e111cfa-892a-4eaf-bdb1-750266e0e6af None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:06.940849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:06.975804 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16999ca9-5fdf-4096-8536-ad2e202c88f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:06.977250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16999ca9-5fdf-4096-8536-ad2e202c88f9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:06.977550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16999ca9-5fdf-4096-8536-ad2e202c88f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:06.977809 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16999ca9-5fdf-4096-8536-ad2e202c88f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:06.978526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16999ca9-5fdf-4096-8536-ad2e202c88f9 None admin] RBAC: Adding query filter params (interface=public, service_id=0bec330bef0547498baaae312f85a005, region_id=RegionOne) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:06.978841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16999ca9-5fdf-4096-8536-ad2e202c88f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:06.980497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16999ca9-5fdf-4096-8536-ad2e202c88f9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:06.991051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 399/782] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:06.995700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 384/783] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Aug 26 15:32:06 2022] GET /identity/v3/endpoints?service_id=0bec330bef0547498baaae312f85a005&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:07.000827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f926870-fbf2-45f1-ae35-4644a32d56b0 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:07.094420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:07.143250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16999ca9-5fdf-4096-8536-ad2e202c88f9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:07.144976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f926870-fbf2-45f1-ae35-4644a32d56b0 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:07.209470 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef0a60-5da2-4f6c-a2da-44936aab7b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:07.210469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef0a60-5da2-4f6c-a2da-44936aab7b77 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:07.213026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef0a60-5da2-4f6c-a2da-44936aab7b77 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:07.213555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef0a60-5da2-4f6c-a2da-44936aab7b77 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:07.214369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef0a60-5da2-4f6c-a2da-44936aab7b77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:07.215999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef0a60-5da2-4f6c-a2da-44936aab7b77 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:07.228002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 400/784] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Aug 26 15:32:06 2022] GET /identity/v3/services => generated 772 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:07.310102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16999ca9-5fdf-4096-8536-ad2e202c88f9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:07.360705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef0a60-5da2-4f6c-a2da-44936aab7b77 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:07.365074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a2fc4-ac07-4795-bff5-ab5e932d0fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:07.365725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a2fc4-ac07-4795-bff5-ab5e932d0fb1 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:07.366008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a2fc4-ac07-4795-bff5-ab5e932d0fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:07.366286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a2fc4-ac07-4795-bff5-ab5e932d0fb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:07.377868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a2fc4-ac07-4795-bff5-ab5e932d0fb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-692a2fc4-ac07-4795-bff5-ab5e932d0fb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:07.383319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:07.384336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:07.385499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 385/785] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:32:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:07.390721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a2fc4-ac07-4795-bff5-ab5e932d0fb1 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:07.436875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:07.515298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef0a60-5da2-4f6c-a2da-44936aab7b77 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:07.535224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a2fc4-ac07-4795-bff5-ab5e932d0fb1 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:07.543699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:07.554025 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0b687-4aab-484f-a4cb-0f082722283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:07.555002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0b687-4aab-484f-a4cb-0f082722283c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:07.555452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0b687-4aab-484f-a4cb-0f082722283c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:07.555926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0b687-4aab-484f-a4cb-0f082722283c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:07.567909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0b687-4aab-484f-a4cb-0f082722283c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-d3f0b687-4aab-484f-a4cb-0f082722283c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:07.573595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:07.574524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:07.575416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:07.582735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 401/786] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:32:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:07.583587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869090c-022c-4ba7-8926-560544d184aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:07.584226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0b687-4aab-484f-a4cb-0f082722283c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:07.584350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869090c-022c-4ba7-8926-560544d184aa None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:07.584517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869090c-022c-4ba7-8926-560544d184aa None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:07.584785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869090c-022c-4ba7-8926-560544d184aa None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:07.585466 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869090c-022c-4ba7-8926-560544d184aa None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:07.585771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869090c-022c-4ba7-8926-560544d184aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:07.587142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869090c-022c-4ba7-8926-560544d184aa None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:07.608486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 386/787] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:32:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:07.608486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869090c-022c-4ba7-8926-560544d184aa None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:07.708946 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0b687-4aab-484f-a4cb-0f082722283c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:07.748660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb19c4d-a54c-40ff-bbdd-a0de24a25fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:07.749526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb19c4d-a54c-40ff-bbdd-a0de24a25fcd None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:07.751883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb19c4d-a54c-40ff-bbdd-a0de24a25fcd None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:07.752325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb19c4d-a54c-40ff-bbdd-a0de24a25fcd None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:07.753213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb19c4d-a54c-40ff-bbdd-a0de24a25fcd None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:07.753675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb19c4d-a54c-40ff-bbdd-a0de24a25fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:07.756500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb19c4d-a54c-40ff-bbdd-a0de24a25fcd None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:07.772605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 402/788] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:32:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:07.778741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb19c4d-a54c-40ff-bbdd-a0de24a25fcd None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:07.796846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:07.807002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869090c-022c-4ba7-8926-560544d184aa None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:07.851676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:07.886116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c9bbb-bc51-4b8e-80e7-aa0223c178a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:07.886763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c9bbb-bc51-4b8e-80e7-aa0223c178a4 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:07.887051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c9bbb-bc51-4b8e-80e7-aa0223c178a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:07.887327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c9bbb-bc51-4b8e-80e7-aa0223c178a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:07.907277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb19c4d-a54c-40ff-bbdd-a0de24a25fcd None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:07.916955 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c9bbb-bc51-4b8e-80e7-aa0223c178a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:07.918669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c9bbb-bc51-4b8e-80e7-aa0223c178a4 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:07.945707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62183d37-3d86-4579-af08-d15fe7c4e301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:07.946360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62183d37-3d86-4579-af08-d15fe7c4e301 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:07.946705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62183d37-3d86-4579-af08-d15fe7c4e301 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:07.946917 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62183d37-3d86-4579-af08-d15fe7c4e301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-df2c9bbb-bc51-4b8e-80e7-aa0223c178a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:07.952032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:07.952971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:07.953873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:07.953873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:07.953873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:07.960538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62183d37-3d86-4579-af08-d15fe7c4e301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:07.961044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 387/789] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:07.961318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62183d37-3d86-4579-af08-d15fe7c4e301 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:07.962323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c9bbb-bc51-4b8e-80e7-aa0223c178a4 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-62183d37-3d86-4579-af08-d15fe7c4e301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:07.978202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:07.979347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:07.980330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:07.980330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:07.980330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:07.980330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 403/790] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:07.992333 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62183d37-3d86-4579-af08-d15fe7c4e301 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:08.092258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:08.096223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62183d37-3d86-4579-af08-d15fe7c4e301 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:08.128569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c9bbb-bc51-4b8e-80e7-aa0223c178a4 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:08.134727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bacdd4-b365-4cc5-8898-6b5a937f0361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:08.135733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bacdd4-b365-4cc5-8898-6b5a937f0361 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:08.136275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bacdd4-b365-4cc5-8898-6b5a937f0361 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:08.136567 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bacdd4-b365-4cc5-8898-6b5a937f0361 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:08.137237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bacdd4-b365-4cc5-8898-6b5a937f0361 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:08.137547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bacdd4-b365-4cc5-8898-6b5a937f0361 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:08.138921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bacdd4-b365-4cc5-8898-6b5a937f0361 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:08.164799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 404/791] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:08.169737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bacdd4-b365-4cc5-8898-6b5a937f0361 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:08.196671 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505020c-df1f-4122-a0cf-6b03c970daad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:08.197286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505020c-df1f-4122-a0cf-6b03c970daad None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:08.197569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505020c-df1f-4122-a0cf-6b03c970daad None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:08.197846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505020c-df1f-4122-a0cf-6b03c970daad None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:08.198495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505020c-df1f-4122-a0cf-6b03c970daad None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:08.198798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505020c-df1f-4122-a0cf-6b03c970daad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:08.200207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505020c-df1f-4122-a0cf-6b03c970daad None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:08.221629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 388/792] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:08.228000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505020c-df1f-4122-a0cf-6b03c970daad None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:08.238880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:08.330373 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bacdd4-b365-4cc5-8898-6b5a937f0361 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:08.370684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60931f-1135-48c2-ad74-3b5eaa0c6efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:08.371484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60931f-1135-48c2-ad74-3b5eaa0c6efb None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:08.371484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60931f-1135-48c2-ad74-3b5eaa0c6efb None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:08.371973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60931f-1135-48c2-ad74-3b5eaa0c6efb None admin] PATH_INFO: `/v3/users/swift` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:08.381587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60931f-1135-48c2-ad74-3b5eaa0c6efb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:08.383573 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60931f-1135-48c2-ad74-3b5eaa0c6efb None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-cd60931f-1135-48c2-ad74-3b5eaa0c6efb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:08.396345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:08.397365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:08.398359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:08.398359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:08.398359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:08.398359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:08.398359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:08.398359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:08.398359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:08.398359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:08.398359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:32:08.398359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:08.398359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 405/793] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:32:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:08.404148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60931f-1135-48c2-ad74-3b5eaa0c6efb None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:08.430667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:08.447620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505020c-df1f-4122-a0cf-6b03c970daad None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:08.512718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6bb1b-2537-44e3-969e-585bff516721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:08.513261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6bb1b-2537-44e3-969e-585bff516721 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:08.513537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6bb1b-2537-44e3-969e-585bff516721 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:08.513809 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6bb1b-2537-44e3-969e-585bff516721 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:08.526173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6bb1b-2537-44e3-969e-585bff516721 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:08.527665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6bb1b-2537-44e3-969e-585bff516721 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-d6c6bb1b-2537-44e3-969e-585bff516721 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:08.545536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:08.546564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:08.547552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:08.547552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:08.547552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:08.547552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:08.547552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:08.547552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:08.547552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:08.547552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:08.547552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:08.547552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:08.548141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 389/794] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Aug 26 15:32:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:08.555292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6bb1b-2537-44e3-969e-585bff516721 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:08.570478 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60931f-1135-48c2-ad74-3b5eaa0c6efb None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:08.633259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ae382-0bc9-4fc7-9536-7837285d5adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:08.633868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae382-0bc9-4fc7-9536-7837285d5adf None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:08.634148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae382-0bc9-4fc7-9536-7837285d5adf None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:08.634421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae382-0bc9-4fc7-9536-7837285d5adf None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:08.635267 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ae382-0bc9-4fc7-9536-7837285d5adf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:08.635577 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ae382-0bc9-4fc7-9536-7837285d5adf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:08.637085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ae382-0bc9-4fc7-9536-7837285d5adf None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:08.671065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 406/795] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Aug 26 15:32:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:08.676175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ae382-0bc9-4fc7-9536-7837285d5adf None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:08.690737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:08.759618 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6bb1b-2537-44e3-969e-585bff516721 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:08.817626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:08.821344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e65219-c2f5-4156-bd56-dd80facdbc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:08.821952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e65219-c2f5-4156-bd56-dd80facdbc52 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:08.822232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e65219-c2f5-4156-bd56-dd80facdbc52 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:08.822504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e65219-c2f5-4156-bd56-dd80facdbc52 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:08.823233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e65219-c2f5-4156-bd56-dd80facdbc52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:08.823536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e65219-c2f5-4156-bd56-dd80facdbc52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:08.824987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e65219-c2f5-4156-bd56-dd80facdbc52 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:08.843627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ae382-0bc9-4fc7-9536-7837285d5adf None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:08.843829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 390/796] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Aug 26 15:32:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:08.849273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea387c74-cd13-4ca8-b61f-688830148ab6 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:08.849559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea387c74-cd13-4ca8-b61f-688830148ab6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:08.849835 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea387c74-cd13-4ca8-b61f-688830148ab6 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:08.851405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 391/797] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:08.853537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea387c74-cd13-4ca8-b61f-688830148ab6 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:08.896089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94116b5e-f860-4035-822f-d270ba478279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:08.896752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94116b5e-f860-4035-822f-d270ba478279 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:08.897033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94116b5e-f860-4035-822f-d270ba478279 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:08.899893 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94116b5e-f860-4035-822f-d270ba478279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:08.913641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94116b5e-f860-4035-822f-d270ba478279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:08.915484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94116b5e-f860-4035-822f-d270ba478279 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-94116b5e-f860-4035-822f-d270ba478279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:08.936998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:08.938926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:08.941361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:08.941361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:08.941361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:08.947891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 407/798] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:08.947891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c8a1-ac77-451a-ac8f-dc33e0148e40 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:08.947891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c8a1-ac77-451a-ac8f-dc33e0148e40 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:08.947891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c8a1-ac77-451a-ac8f-dc33e0148e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:08.968149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-5a26c8a1-ac77-451a-ac8f-dc33e0148e40 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:08.969353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5a26c8a1-ac77-451a-ac8f-dc33e0148e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:08.970479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea387c74-cd13-4ca8-b61f-688830148ab6 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:08.997818 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:09.008760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c982fd1-f797-4e45-b117-c51ec3f83fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:09.009606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c982fd1-f797-4e45-b117-c51ec3f83fd6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.009920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c982fd1-f797-4e45-b117-c51ec3f83fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.010157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c982fd1-f797-4e45-b117-c51ec3f83fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.020440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c982fd1-f797-4e45-b117-c51ec3f83fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:09.021895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c982fd1-f797-4e45-b117-c51ec3f83fd6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-7c982fd1-f797-4e45-b117-c51ec3f83fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:09.037987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:09.039163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:09.040331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:09.040331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:09.040331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:09.042318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 392/799] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:09.042823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 408/800] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:09.044693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c982fd1-f797-4e45-b117-c51ec3f83fd6 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:09.053234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b681d-f909-41f4-847f-795b527c2aec None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.053531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b681d-f909-41f4-847f-795b527c2aec None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.053809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b681d-f909-41f4-847f-795b527c2aec None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.055918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 409/801] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:09 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:09.055918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b681d-f909-41f4-847f-795b527c2aec None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:09.140044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:09.160080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c982fd1-f797-4e45-b117-c51ec3f83fd6 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:09.198247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e069c5d-d1d8-41ac-83cf-bf85869a3bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:09.198883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e069c5d-d1d8-41ac-83cf-bf85869a3bc1 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.199161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e069c5d-d1d8-41ac-83cf-bf85869a3bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.199434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e069c5d-d1d8-41ac-83cf-bf85869a3bc1 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.200214 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e069c5d-d1d8-41ac-83cf-bf85869a3bc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:09.200517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e069c5d-d1d8-41ac-83cf-bf85869a3bc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:09.202082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b681d-f909-41f4-847f-795b527c2aec None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:09.202442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e069c5d-d1d8-41ac-83cf-bf85869a3bc1 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:09.216270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 393/802] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:09.217657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ce340-c3f5-4f96-8224-0b8e578c2e5d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.217943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ce340-c3f5-4f96-8224-0b8e578c2e5d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.218215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ce340-c3f5-4f96-8224-0b8e578c2e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.237231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-935ce340-c3f5-4f96-8224-0b8e578c2e5d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:09.238415 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-935ce340-c3f5-4f96-8224-0b8e578c2e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:09.245909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0b162-cc05-4023-9df2-229845ebe271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:09.247301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0b162-cc05-4023-9df2-229845ebe271 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.247798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0b162-cc05-4023-9df2-229845ebe271 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.248320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0b162-cc05-4023-9df2-229845ebe271 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.249655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0b162-cc05-4023-9df2-229845ebe271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:09.250186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0b162-cc05-4023-9df2-229845ebe271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:09.252482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0b162-cc05-4023-9df2-229845ebe271 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:09.269830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 410/803] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:09.270943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0b162-cc05-4023-9df2-229845ebe271 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:09.276575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:09.334164 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 394/804] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:09.335424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ce340-c3f5-4f96-8224-0b8e578c2e5d None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:09.395469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0b162-cc05-4023-9df2-229845ebe271 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:09.417032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:09.433395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7162d28-0781-4da6-9bde-337a07d9b978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:09.434365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7162d28-0781-4da6-9bde-337a07d9b978 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.434879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7162d28-0781-4da6-9bde-337a07d9b978 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.435358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7162d28-0781-4da6-9bde-337a07d9b978 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.443419 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7162d28-0781-4da6-9bde-337a07d9b978 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:09.445136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7162d28-0781-4da6-9bde-337a07d9b978 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:09.450702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ce340-c3f5-4f96-8224-0b8e578c2e5d None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-b7162d28-0781-4da6-9bde-337a07d9b978 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:09.457887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:09.458793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:09.459752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:09.459752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:09.459752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:09.459752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:09.459752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:09.459752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:09.459752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:09.459752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:09.462155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 411/805] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:09.463194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7162d28-0781-4da6-9bde-337a07d9b978 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:09.493358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87cf934-5a84-47ed-915d-034b4fc3e9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:09.494019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cf934-5a84-47ed-915d-034b4fc3e9c4 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.494304 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cf934-5a84-47ed-915d-034b4fc3e9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.494580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cf934-5a84-47ed-915d-034b4fc3e9c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.501625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87cf934-5a84-47ed-915d-034b4fc3e9c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:09.503895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87cf934-5a84-47ed-915d-034b4fc3e9c4 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-a87cf934-5a84-47ed-915d-034b4fc3e9c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:09.516056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:09.517010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:09.517990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:09.517990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:09.517990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:09.517990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:09.517990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:09.517990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:09.517990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:09.517990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:09.517990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 395/806] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:09.518491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87cf934-5a84-47ed-915d-034b4fc3e9c4 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:09.546524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:09.595107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7162d28-0781-4da6-9bde-337a07d9b978 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:09.634698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87cf934-5a84-47ed-915d-034b4fc3e9c4 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:09.639466 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cf3c4-2d60-40e7-8524-43485d86de6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:09.640132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cf3c4-2d60-40e7-8524-43485d86de6b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.640420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cf3c4-2d60-40e7-8524-43485d86de6b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.640697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cf3c4-2d60-40e7-8524-43485d86de6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.654153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cf3c4-2d60-40e7-8524-43485d86de6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-288cf3c4-2d60-40e7-8524-43485d86de6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:09.661880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:09.662858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:09.663725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:09.664431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 412/807] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:32:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:09.664961 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cf3c4-2d60-40e7-8524-43485d86de6b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:09.673502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31070d26-0ad0-49d5-9afd-97d3a465e6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:09.674135 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31070d26-0ad0-49d5-9afd-97d3a465e6b0 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.675089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31070d26-0ad0-49d5-9afd-97d3a465e6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.675362 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31070d26-0ad0-49d5-9afd-97d3a465e6b0 None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.676087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31070d26-0ad0-49d5-9afd-97d3a465e6b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:09.676393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31070d26-0ad0-49d5-9afd-97d3a465e6b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:09.677820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31070d26-0ad0-49d5-9afd-97d3a465e6b0 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:09.691503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 396/808] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:09.692410 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31070d26-0ad0-49d5-9afd-97d3a465e6b0 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:09.714565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:09.774557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cf3c4-2d60-40e7-8524-43485d86de6b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:09.781965 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:09.796156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31070d26-0ad0-49d5-9afd-97d3a465e6b0 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:09.807143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fffba1-cd85-4f61-be3f-bfb3cb2cc5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:09.807780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fffba1-cd85-4f61-be3f-bfb3cb2cc5e1 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.808096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fffba1-cd85-4f61-be3f-bfb3cb2cc5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.808370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fffba1-cd85-4f61-be3f-bfb3cb2cc5e1 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.809050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fffba1-cd85-4f61-be3f-bfb3cb2cc5e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:09.809362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fffba1-cd85-4f61-be3f-bfb3cb2cc5e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:09.810824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fffba1-cd85-4f61-be3f-bfb3cb2cc5e1 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:09.824085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 413/809] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:09.825474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fffba1-cd85-4f61-be3f-bfb3cb2cc5e1 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:09.835290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88c0bf-6174-493e-ae50-ad9fa161aa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:09.835958 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88c0bf-6174-493e-ae50-ad9fa161aa7e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.836241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88c0bf-6174-493e-ae50-ad9fa161aa7e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.836514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88c0bf-6174-493e-ae50-ad9fa161aa7e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.837110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88c0bf-6174-493e-ae50-ad9fa161aa7e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:09.838423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88c0bf-6174-493e-ae50-ad9fa161aa7e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-bf88c0bf-6174-493e-ae50-ad9fa161aa7e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:09.847143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:09.848251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:09.849152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 397/810] 10.222.0.98 () {56 vars in 1129 bytes} [Fri Aug 26 15:32:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:09.849988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88c0bf-6174-493e-ae50-ad9fa161aa7e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:09.923311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:09.926850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fffba1-cd85-4f61-be3f-bfb3cb2cc5e1 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:09.952750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88c0bf-6174-493e-ae50-ad9fa161aa7e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:09.960468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fe7bc-302a-48b3-a94d-f61ca8a8c562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:09.961072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fe7bc-302a-48b3-a94d-f61ca8a8c562 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.961352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fe7bc-302a-48b3-a94d-f61ca8a8c562 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.961632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fe7bc-302a-48b3-a94d-f61ca8a8c562 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.962296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fe7bc-302a-48b3-a94d-f61ca8a8c562 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:09.962608 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fe7bc-302a-48b3-a94d-f61ca8a8c562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:09.964007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fe7bc-302a-48b3-a94d-f61ca8a8c562 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:09.972738 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 414/811] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:32:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:09.973890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a34cb-21fc-4e77-9ddf-e1a814dfdcba None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.974182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a34cb-21fc-4e77-9ddf-e1a814dfdcba None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.974462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a34cb-21fc-4e77-9ddf-e1a814dfdcba None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.975661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 415/812] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:09.979018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a34cb-21fc-4e77-9ddf-e1a814dfdcba None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:09.989688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3a72d-78f2-414a-a1b6-96c0793a6afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:09.990350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3a72d-78f2-414a-a1b6-96c0793a6afa None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:09.990637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3a72d-78f2-414a-a1b6-96c0793a6afa None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:09.990918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3a72d-78f2-414a-a1b6-96c0793a6afa None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/1fae47d6b7a7472a9d94636bb126f802/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:09.993270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3a72d-78f2-414a-a1b6-96c0793a6afa None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=1fae47d6b7a7472a9d94636bb126f802, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:09.994709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3a72d-78f2-414a-a1b6-96c0793a6afa None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:10.054421 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 398/813] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:32:09 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/1fae47d6b7a7472a9d94636bb126f802/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:32:10.055249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3a72d-78f2-414a-a1b6-96c0793a6afa None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:10.069007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:10.082386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a34cb-21fc-4e77-9ddf-e1a814dfdcba None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:10.125305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9df484-d188-4d88-9306-f366d65983d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:10.125992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9df484-d188-4d88-9306-f366d65983d9 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:10.126269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9df484-d188-4d88-9306-f366d65983d9 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:10.126549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9df484-d188-4d88-9306-f366d65983d9 None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/b24a6d0c1dbe49a18b3b8b6732a938be/roles/9dc2d706af5146e0b8d62ef94c74906d` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:10.143473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9df484-d188-4d88-9306-f366d65983d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=b24a6d0c1dbe49a18b3b8b6732a938be, role_id=9dc2d706af5146e0b8d62ef94c74906d)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:10.146206 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9df484-d188-4d88-9306-f366d65983d9 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:10.176212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3a72d-78f2-414a-a1b6-96c0793a6afa None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:10.215994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 416/814] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:32:09 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/b24a6d0c1dbe49a18b3b8b6732a938be/roles/9dc2d706af5146e0b8d62ef94c74906d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:32:10.216859 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9df484-d188-4d88-9306-f366d65983d9 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:10.223762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43549a8a-11f4-4db0-b27b-d31b4c07ac14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:10.224432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43549a8a-11f4-4db0-b27b-d31b4c07ac14 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:10.226288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43549a8a-11f4-4db0-b27b-d31b4c07ac14 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:10.226288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43549a8a-11f4-4db0-b27b-d31b4c07ac14 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:10.226288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43549a8a-11f4-4db0-b27b-d31b4c07ac14 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:10.226288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43549a8a-11f4-4db0-b27b-d31b4c07ac14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:10.227360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43549a8a-11f4-4db0-b27b-d31b4c07ac14 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:10.235989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 399/815] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Aug 26 15:32:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:10.235989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367117b6-6b42-4c00-845f-638e2105a3f3 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:10.235989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367117b6-6b42-4c00-845f-638e2105a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:10.236381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367117b6-6b42-4c00-845f-638e2105a3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:10.252310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:10.261079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-367117b6-6b42-4c00-845f-638e2105a3f3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:10.262937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-367117b6-6b42-4c00-845f-638e2105a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:10.359225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9df484-d188-4d88-9306-f366d65983d9 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:10.372813 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 400/816] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:10.373669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367117b6-6b42-4c00-845f-638e2105a3f3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:10.378571 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:10.399892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf752b-0e92-402a-bba4-48e2acf72db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:10.400913 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf752b-0e92-402a-bba4-48e2acf72db9 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:10.401441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf752b-0e92-402a-bba4-48e2acf72db9 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:10.401792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf752b-0e92-402a-bba4-48e2acf72db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:10.417183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf752b-0e92-402a-bba4-48e2acf72db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:10.419014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf752b-0e92-402a-bba4-48e2acf72db9 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-d9cf752b-0e92-402a-bba4-48e2acf72db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:10.445821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:10.446869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:10.447791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:10.447791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:10.447791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:10.449388 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 417/817] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:10.457795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf752b-0e92-402a-bba4-48e2acf72db9 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:10.498168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367117b6-6b42-4c00-845f-638e2105a3f3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:10.550812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf9a64-e7e1-4008-854c-eee85b1878e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:10.551434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf9a64-e7e1-4008-854c-eee85b1878e6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:10.551717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf9a64-e7e1-4008-854c-eee85b1878e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:10.552022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf9a64-e7e1-4008-854c-eee85b1878e6 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:10.552706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf9a64-e7e1-4008-854c-eee85b1878e6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:10.553015 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf9a64-e7e1-4008-854c-eee85b1878e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:10.554393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf9a64-e7e1-4008-854c-eee85b1878e6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:10.571895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 401/818] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Aug 26 15:32:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:10.571895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf9a64-e7e1-4008-854c-eee85b1878e6 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:10.604228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:10.676799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf752b-0e92-402a-bba4-48e2acf72db9 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:10.701244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:10.715130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a9d7f-93d1-4521-afae-bebc295d7954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:10.716254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a9d7f-93d1-4521-afae-bebc295d7954 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:10.716735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a9d7f-93d1-4521-afae-bebc295d7954 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:10.717223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a9d7f-93d1-4521-afae-bebc295d7954 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:10.718057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a9d7f-93d1-4521-afae-bebc295d7954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:10.718553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a9d7f-93d1-4521-afae-bebc295d7954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:10.720164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a9d7f-93d1-4521-afae-bebc295d7954 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:10.732872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 418/819] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:10.734051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a9d7f-93d1-4521-afae-bebc295d7954 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:10.744765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf9a64-e7e1-4008-854c-eee85b1878e6 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:10.781358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e534edc-639c-4719-9bf8-151c227773ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:10.782192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e534edc-639c-4719-9bf8-151c227773ab None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:10.782687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e534edc-639c-4719-9bf8-151c227773ab None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:10.783132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e534edc-639c-4719-9bf8-151c227773ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:10.784578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e534edc-639c-4719-9bf8-151c227773ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:10.786348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e534edc-639c-4719-9bf8-151c227773ab None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:10.844261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a9d7f-93d1-4521-afae-bebc295d7954 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:10.868587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:10.879042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e059dc-0214-4fe0-84b0-66eb9faa4c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:10.879669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e059dc-0214-4fe0-84b0-66eb9faa4c6f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:10.879985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e059dc-0214-4fe0-84b0-66eb9faa4c6f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:10.880265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e059dc-0214-4fe0-84b0-66eb9faa4c6f None admin] PATH_INFO: `/v3/services/network` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:10.880844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e059dc-0214-4fe0-84b0-66eb9faa4c6f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:10.882207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e059dc-0214-4fe0-84b0-66eb9faa4c6f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-92e059dc-0214-4fe0-84b0-66eb9faa4c6f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:10.890951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:10.892551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:10.894323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 419/820] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:10 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:10.898046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e059dc-0214-4fe0-84b0-66eb9faa4c6f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:10.911949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 402/821] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:32:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:10.914146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e534edc-639c-4719-9bf8-151c227773ab None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:11.010072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e059dc-0214-4fe0-84b0-66eb9faa4c6f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:11.051959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916cffe-32f2-4b7a-aedc-c9a39060cb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:11.052604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916cffe-32f2-4b7a-aedc-c9a39060cb1c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:11.052889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916cffe-32f2-4b7a-aedc-c9a39060cb1c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:11.053173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916cffe-32f2-4b7a-aedc-c9a39060cb1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:11.061919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916cffe-32f2-4b7a-aedc-c9a39060cb1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:11.063413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916cffe-32f2-4b7a-aedc-c9a39060cb1c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-a916cffe-32f2-4b7a-aedc-c9a39060cb1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:11.076627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:11.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:11.078588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:11.078588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:11.078588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:11.078588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:11.078588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:11.078588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:11.078588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:11.078588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:11.078588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:32:11.078588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:11.078588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 420/822] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Aug 26 15:32:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:11.083478 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916cffe-32f2-4b7a-aedc-c9a39060cb1c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:11.152529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e534edc-639c-4719-9bf8-151c227773ab None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:11.163496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:11.188822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:11.204693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bd132-9a8a-42b7-9c8e-51270bfd470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:11.205986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bd132-9a8a-42b7-9c8e-51270bfd470d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:11.206446 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bd132-9a8a-42b7-9c8e-51270bfd470d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:11.208031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bd132-9a8a-42b7-9c8e-51270bfd470d None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:11.208031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bd132-9a8a-42b7-9c8e-51270bfd470d None admin] RBAC: Adding query filter params (name=network) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:11.208734 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bd132-9a8a-42b7-9c8e-51270bfd470d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:11.210809 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bd132-9a8a-42b7-9c8e-51270bfd470d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:11.216178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 403/823] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:32:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:11.223147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bd132-9a8a-42b7-9c8e-51270bfd470d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:11.236523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916cffe-32f2-4b7a-aedc-c9a39060cb1c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:11.279478 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970e867-ade9-4eaf-a3df-77c7acc3f619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:11.280225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970e867-ade9-4eaf-a3df-77c7acc3f619 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:11.280609 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970e867-ade9-4eaf-a3df-77c7acc3f619 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:11.280985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970e867-ade9-4eaf-a3df-77c7acc3f619 None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:11.281824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970e867-ade9-4eaf-a3df-77c7acc3f619 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:11.282346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970e867-ade9-4eaf-a3df-77c7acc3f619 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:11.283905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970e867-ade9-4eaf-a3df-77c7acc3f619 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:11.306100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 421/824] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Aug 26 15:32:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:11.311681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970e867-ade9-4eaf-a3df-77c7acc3f619 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:11.362537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bd132-9a8a-42b7-9c8e-51270bfd470d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:11.407894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3785738-85b9-468f-9749-c454f9506ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:11.409624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3785738-85b9-468f-9749-c454f9506ba7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:11.410584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3785738-85b9-468f-9749-c454f9506ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:11.411187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3785738-85b9-468f-9749-c454f9506ba7 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:11.412158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3785738-85b9-468f-9749-c454f9506ba7 None admin] RBAC: Adding query filter params (type=network) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:11.412929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3785738-85b9-468f-9749-c454f9506ba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:11.414929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3785738-85b9-468f-9749-c454f9506ba7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:11.422058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 404/825] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:32:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:11.428809 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3785738-85b9-468f-9749-c454f9506ba7 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:11.435015 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970e867-ade9-4eaf-a3df-77c7acc3f619 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:11.436731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:11.473163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee55fd-4601-49e2-b140-9e8c3bce1f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:11.474044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee55fd-4601-49e2-b140-9e8c3bce1f79 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:11.474648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee55fd-4601-49e2-b140-9e8c3bce1f79 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:11.475078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee55fd-4601-49e2-b140-9e8c3bce1f79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:11.548053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:11.554198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee55fd-4601-49e2-b140-9e8c3bce1f79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:11.555845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee55fd-4601-49e2-b140-9e8c3bce1f79 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:11.671535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3785738-85b9-468f-9749-c454f9506ba7 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:11.671915 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 422/826] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 15:32:11.673397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee55fd-4601-49e2-b140-9e8c3bce1f79 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:11.675899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:11.764577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876661bb-6fab-424b-aff5-dd332248b088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:11.765316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876661bb-6fab-424b-aff5-dd332248b088 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:11.765674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876661bb-6fab-424b-aff5-dd332248b088 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:11.766042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876661bb-6fab-424b-aff5-dd332248b088 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:11.766778 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876661bb-6fab-424b-aff5-dd332248b088 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:11.768787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876661bb-6fab-424b-aff5-dd332248b088 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:11.784623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee55fd-4601-49e2-b140-9e8c3bce1f79 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:11.822989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f83705-027e-4027-af22-d0cce939957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:11.823938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f83705-027e-4027-af22-d0cce939957c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:11.824487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f83705-027e-4027-af22-d0cce939957c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:11.824967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f83705-027e-4027-af22-d0cce939957c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:11.837424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f83705-027e-4027-af22-d0cce939957c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:11.839122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f83705-027e-4027-af22-d0cce939957c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-f4f83705-027e-4027-af22-d0cce939957c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:11.855226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:11.856707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:11.857622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:11.857622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:11.857622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:11.863288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 423/827] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:11.865448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f83705-027e-4027-af22-d0cce939957c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:11.887398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 405/828] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:32:11 2022] POST /identity/v3/endpoints => generated 337 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:11.947461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:12.008647 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f83705-027e-4027-af22-d0cce939957c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:12.065997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfb4f8-c84f-4768-8da5-2c6b366bf5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:12.066644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfb4f8-c84f-4768-8da5-2c6b366bf5c6 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:12.066934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfb4f8-c84f-4768-8da5-2c6b366bf5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:12.067205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfb4f8-c84f-4768-8da5-2c6b366bf5c6 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:12.067841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfb4f8-c84f-4768-8da5-2c6b366bf5c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:12.068173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfb4f8-c84f-4768-8da5-2c6b366bf5c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:12.069545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfb4f8-c84f-4768-8da5-2c6b366bf5c6 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:12.083895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 424/829] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:12.090518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876661bb-6fab-424b-aff5-dd332248b088 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:12.201156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876661bb-6fab-424b-aff5-dd332248b088 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:12.212518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:12.239566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbd2c8-1afc-425c-8e84-3e81e807e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:12.240529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbd2c8-1afc-425c-8e84-3e81e807e39b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:12.240960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbd2c8-1afc-425c-8e84-3e81e807e39b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:12.241382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbd2c8-1afc-425c-8e84-3e81e807e39b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:12.248469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbd2c8-1afc-425c-8e84-3e81e807e39b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:12.250253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbd2c8-1afc-425c-8e84-3e81e807e39b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-5efbd2c8-1afc-425c-8e84-3e81e807e39b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:12.264491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:12.265607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:12.266807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:12.266807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:12.266807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:12.266807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:12.266807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:12.266807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:12.266807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:12.266807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:12.269081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 406/830] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:12.275043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfb4f8-c84f-4768-8da5-2c6b366bf5c6 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:12.391308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8cf9-759e-4b1c-b9f3-f8c47bfb9748 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:12.391827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8cf9-759e-4b1c-b9f3-f8c47bfb9748 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:12.392124 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8cf9-759e-4b1c-b9f3-f8c47bfb9748 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:12.393668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 407/831] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:12.394595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfb4f8-c84f-4768-8da5-2c6b366bf5c6 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:12.399667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0cf0-4492-4f57-a61e-926d096b7e5f None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:12.400244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0cf0-4492-4f57-a61e-926d096b7e5f None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:12.400749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0cf0-4492-4f57-a61e-926d096b7e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:12.406815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:12.420838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c8be0cf0-4492-4f57-a61e-926d096b7e5f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:12.422412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c8be0cf0-4492-4f57-a61e-926d096b7e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:12.432406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e3b49-5fbd-4284-b2ed-6f124615199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:12.433177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e3b49-5fbd-4284-b2ed-6f124615199a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:12.433402 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e3b49-5fbd-4284-b2ed-6f124615199a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:12.433680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e3b49-5fbd-4284-b2ed-6f124615199a None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:12.434427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e3b49-5fbd-4284-b2ed-6f124615199a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:12.434735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e3b49-5fbd-4284-b2ed-6f124615199a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:12.436265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e3b49-5fbd-4284-b2ed-6f124615199a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:12.448829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 425/832] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:12.450163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c41d64-f79c-4f0c-91ba-b787ac182a94 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:12.450448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c41d64-f79c-4f0c-91ba-b787ac182a94 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:12.450725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c41d64-f79c-4f0c-91ba-b787ac182a94 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:12.452087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 426/833] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:12.454348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c41d64-f79c-4f0c-91ba-b787ac182a94 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:12.501403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 408/834] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:12.503320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036915c3-fcf2-46cd-ab75-0eb447530b6a None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:12.503613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036915c3-fcf2-46cd-ab75-0eb447530b6a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:12.503937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036915c3-fcf2-46cd-ab75-0eb447530b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:12.527579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-036915c3-fcf2-46cd-ab75-0eb447530b6a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:12.528815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-036915c3-fcf2-46cd-ab75-0eb447530b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:12.584170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c41d64-f79c-4f0c-91ba-b787ac182a94 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:12.598504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 409/835] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:12.630990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad963c-6f3e-423c-b397-579e876329a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:12.631597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad963c-6f3e-423c-b397-579e876329a7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:12.631890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad963c-6f3e-423c-b397-579e876329a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:12.632182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad963c-6f3e-423c-b397-579e876329a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:12.633012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad963c-6f3e-423c-b397-579e876329a7 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=d1b01c21c38949938141eefdc4939385) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:12.633326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad963c-6f3e-423c-b397-579e876329a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:12.634810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad963c-6f3e-423c-b397-579e876329a7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:12.641305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 427/836] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:12 2022] GET /identity/v3/role_assignments?user.id=d1b01c21c38949938141eefdc4939385&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=6ba5a5add2174983be37258aad1ee404 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:12.776257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036915c3-fcf2-46cd-ab75-0eb447530b6a None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:12.805961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:12.881819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad963c-6f3e-423c-b397-579e876329a7 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:12.884651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036915c3-fcf2-46cd-ab75-0eb447530b6a None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:12.918883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:12.919080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ad033-b682-4b20-b1e1-850ada5b8a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:12.919641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ad033-b682-4b20-b1e1-850ada5b8a94 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:12.919951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ad033-b682-4b20-b1e1-850ada5b8a94 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:12.920229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ad033-b682-4b20-b1e1-850ada5b8a94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:12.931085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ad033-b682-4b20-b1e1-850ada5b8a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-3d4ad033-b682-4b20-b1e1-850ada5b8a94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:12.935900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:12.936972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:12.937828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 410/837] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:32:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:12.945622 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ad033-b682-4b20-b1e1-850ada5b8a94 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:13.001121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad963c-6f3e-423c-b397-579e876329a7 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:13.046791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a886b4-d453-4ea2-be32-b3b5b44ceed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:13.048224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a886b4-d453-4ea2-be32-b3b5b44ceed6 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.048794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a886b4-d453-4ea2-be32-b3b5b44ceed6 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.049544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a886b4-d453-4ea2-be32-b3b5b44ceed6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.054669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:13.063524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a886b4-d453-4ea2-be32-b3b5b44ceed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:13.068587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ad033-b682-4b20-b1e1-850ada5b8a94 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-75a886b4-d453-4ea2-be32-b3b5b44ceed6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:13.069754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:13.072405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:32:13.073484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:13.075548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 428/838] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:32:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:13.081669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a886b4-d453-4ea2-be32-b3b5b44ceed6 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:13.119624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5188f-a54e-4f6e-b131-0821d6b111af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:13.121031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5188f-a54e-4f6e-b131-0821d6b111af None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.121534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5188f-a54e-4f6e-b131-0821d6b111af None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.122015 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5188f-a54e-4f6e-b131-0821d6b111af None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.122888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5188f-a54e-4f6e-b131-0821d6b111af None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:13.123398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5188f-a54e-4f6e-b131-0821d6b111af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:13.125015 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5188f-a54e-4f6e-b131-0821d6b111af None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:13.134718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 411/839] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:32:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:13.143073 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5188f-a54e-4f6e-b131-0821d6b111af None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:13.183228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:13.190120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a886b4-d453-4ea2-be32-b3b5b44ceed6 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:13.226658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cc8f2-8e6f-471a-ba7e-f1d786f8bb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:13.227325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cc8f2-8e6f-471a-ba7e-f1d786f8bb72 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.227613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cc8f2-8e6f-471a-ba7e-f1d786f8bb72 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.227984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cc8f2-8e6f-471a-ba7e-f1d786f8bb72 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.228664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cc8f2-8e6f-471a-ba7e-f1d786f8bb72 None admin] RBAC: Adding query filter params (name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:13.228974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cc8f2-8e6f-471a-ba7e-f1d786f8bb72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:13.230592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cc8f2-8e6f-471a-ba7e-f1d786f8bb72 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:13.244046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 429/840] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:32:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:13.244963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cc8f2-8e6f-471a-ba7e-f1d786f8bb72 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:13.273309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5188f-a54e-4f6e-b131-0821d6b111af None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:13.324592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591ea80-7860-448c-91cd-d6558fd6aaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:13.325585 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591ea80-7860-448c-91cd-d6558fd6aaff None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.326105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591ea80-7860-448c-91cd-d6558fd6aaff None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.327194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591ea80-7860-448c-91cd-d6558fd6aaff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.343499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591ea80-7860-448c-91cd-d6558fd6aaff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:13.347507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591ea80-7860-448c-91cd-d6558fd6aaff None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:13.355346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cc8f2-8e6f-471a-ba7e-f1d786f8bb72 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-b591ea80-7860-448c-91cd-d6558fd6aaff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:13.368150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:13.369185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:13.370181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:13.370181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:13.370181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:13.377606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 412/841] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:13.378596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591ea80-7860-448c-91cd-d6558fd6aaff None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:13.383209 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:13.400748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4de5e4-31c0-4b22-9efb-3f731f246f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:13.401459 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de5e4-31c0-4b22-9efb-3f731f246f9a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.401724 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de5e4-31c0-4b22-9efb-3f731f246f9a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.402118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de5e4-31c0-4b22-9efb-3f731f246f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.414548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4de5e4-31c0-4b22-9efb-3f731f246f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:13.416152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4de5e4-31c0-4b22-9efb-3f731f246f9a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-6b4de5e4-31c0-4b22-9efb-3f731f246f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:13.438816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:13.440953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:13.441865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:13.441865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:13.441865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:13.447228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 430/842] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:13.448533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4de5e4-31c0-4b22-9efb-3f731f246f9a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:13.461530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:13.511237 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591ea80-7860-448c-91cd-d6558fd6aaff None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:13.547145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e047ec-3130-427c-957d-1a748ebda2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:13.547745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e047ec-3130-427c-957d-1a748ebda2f4 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.548058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e047ec-3130-427c-957d-1a748ebda2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.548332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e047ec-3130-427c-957d-1a748ebda2f4 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.549979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e047ec-3130-427c-957d-1a748ebda2f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:13.550296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e047ec-3130-427c-957d-1a748ebda2f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:13.551657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e047ec-3130-427c-957d-1a748ebda2f4 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:13.563196 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 413/843] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:13.564365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4de5e4-31c0-4b22-9efb-3f731f246f9a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:13.564519 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5c725-e414-4562-bf33-e590d3d5a1fa None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.564800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5c725-e414-4562-bf33-e590d3d5a1fa None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.565073 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5c725-e414-4562-bf33-e590d3d5a1fa None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.566528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 414/844] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:13.570202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5c725-e414-4562-bf33-e590d3d5a1fa None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:13.610139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea0264-ef91-43d0-9799-a088237a82bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:13.610750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0264-ef91-43d0-9799-a088237a82bc None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.611595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0264-ef91-43d0-9799-a088237a82bc None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.611893 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0264-ef91-43d0-9799-a088237a82bc None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.612555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea0264-ef91-43d0-9799-a088237a82bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:13.612875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea0264-ef91-43d0-9799-a088237a82bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:13.614237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea0264-ef91-43d0-9799-a088237a82bc None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:13.630829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 431/845] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:13.632622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b452c2b-7c4e-4dd5-87da-a6c4f0718bb0 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.632914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b452c2b-7c4e-4dd5-87da-a6c4f0718bb0 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.633191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b452c2b-7c4e-4dd5-87da-a6c4f0718bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.640866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:13.650461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0b452c2b-7c4e-4dd5-87da-a6c4f0718bb0 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:13.651916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0b452c2b-7c4e-4dd5-87da-a6c4f0718bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:13.687442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5c725-e414-4562-bf33-e590d3d5a1fa None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:13.716419 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:13.722973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 432/846] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:13.723889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b452c2b-7c4e-4dd5-87da-a6c4f0718bb0 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:13.733575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f234f-80e4-4ef8-b15d-3598a4d9bec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:13.734817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f234f-80e4-4ef8-b15d-3598a4d9bec8 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.735668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f234f-80e4-4ef8-b15d-3598a4d9bec8 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.736205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f234f-80e4-4ef8-b15d-3598a4d9bec8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.746478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f234f-80e4-4ef8-b15d-3598a4d9bec8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:13.748604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f234f-80e4-4ef8-b15d-3598a4d9bec8 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-d23f234f-80e4-4ef8-b15d-3598a4d9bec8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:13.763352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:13.764366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:13.765336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:13.765336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:13.765336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:13.765336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:13.765336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:13.765336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:13.765336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:13.765336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:13.765336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:32:13.765336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:13.768441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 415/847] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:32:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:13.775897 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f234f-80e4-4ef8-b15d-3598a4d9bec8 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:13.843464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b452c2b-7c4e-4dd5-87da-a6c4f0718bb0 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:13.881574 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf2695-bb71-454b-8f68-78a2eb21669a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:13.882158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf2695-bb71-454b-8f68-78a2eb21669a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.882686 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf2695-bb71-454b-8f68-78a2eb21669a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.882762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf2695-bb71-454b-8f68-78a2eb21669a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.890066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf2695-bb71-454b-8f68-78a2eb21669a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:13.891506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf2695-bb71-454b-8f68-78a2eb21669a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-ebbf2695-bb71-454b-8f68-78a2eb21669a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:13.904949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:13.905902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:13.906864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:13.906864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:13.906864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:13.906864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:13.906864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:13.906864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:13.906864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:13.906864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:13.906864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:13.906864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:13.909026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f234f-80e4-4ef8-b15d-3598a4d9bec8 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:13.911242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 433/848] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Aug 26 15:32:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:13.912702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf2695-bb71-454b-8f68-78a2eb21669a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:13.938619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:13.955906 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11205b-6d43-4e37-849d-0c94540ca090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:13.956777 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11205b-6d43-4e37-849d-0c94540ca090 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:13.958129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11205b-6d43-4e37-849d-0c94540ca090 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:13.958129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11205b-6d43-4e37-849d-0c94540ca090 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:13.959093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11205b-6d43-4e37-849d-0c94540ca090 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:13.959579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11205b-6d43-4e37-849d-0c94540ca090 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:13.962046 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11205b-6d43-4e37-849d-0c94540ca090 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:13.974380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:13.983924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 416/849] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Aug 26 15:32:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:13.992282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11205b-6d43-4e37-849d-0c94540ca090 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:14.033368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf2695-bb71-454b-8f68-78a2eb21669a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:14.068984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4e1b9-ecef-4f1d-85a9-e786da3f21e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:14.069752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4e1b9-ecef-4f1d-85a9-e786da3f21e3 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:14.070039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4e1b9-ecef-4f1d-85a9-e786da3f21e3 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:14.070348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4e1b9-ecef-4f1d-85a9-e786da3f21e3 None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:14.071091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4e1b9-ecef-4f1d-85a9-e786da3f21e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:14.071409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4e1b9-ecef-4f1d-85a9-e786da3f21e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:14.072859 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4e1b9-ecef-4f1d-85a9-e786da3f21e3 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:14.093601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 434/850] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Aug 26 15:32:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:14.099481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4e1b9-ecef-4f1d-85a9-e786da3f21e3 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:14.103514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11205b-6d43-4e37-849d-0c94540ca090 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:14.143974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a324c-08ea-461d-8b44-d1599b06306e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:14.144603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a324c-08ea-461d-8b44-d1599b06306e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:14.144979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a324c-08ea-461d-8b44-d1599b06306e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:14.145263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a324c-08ea-461d-8b44-d1599b06306e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:14.199922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4e1b9-ecef-4f1d-85a9-e786da3f21e3 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:14.206650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:14.216440 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:14.231339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a324c-08ea-461d-8b44-d1599b06306e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:14.232817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a324c-08ea-461d-8b44-d1599b06306e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:14.234845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04e276-4c36-43ce-acdb-16d7701cf5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:14.235449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04e276-4c36-43ce-acdb-16d7701cf5ba None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:14.235734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04e276-4c36-43ce-acdb-16d7701cf5ba None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:14.236600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04e276-4c36-43ce-acdb-16d7701cf5ba None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:14.237183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04e276-4c36-43ce-acdb-16d7701cf5ba None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:14.238549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04e276-4c36-43ce-acdb-16d7701cf5ba None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-2f04e276-4c36-43ce-acdb-16d7701cf5ba None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:14.247443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:14.248580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:14.249414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 435/851] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:32:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:14.250248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04e276-4c36-43ce-acdb-16d7701cf5ba None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:14.348265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 417/852] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 15:32:14.349478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a324c-08ea-461d-8b44-d1599b06306e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:14.368407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04e276-4c36-43ce-acdb-16d7701cf5ba None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:14.402941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcb66d-8e62-4c2b-ae7e-0264a7608013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:14.403538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcb66d-8e62-4c2b-ae7e-0264a7608013 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:14.403825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcb66d-8e62-4c2b-ae7e-0264a7608013 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:14.404137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcb66d-8e62-4c2b-ae7e-0264a7608013 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:14.434886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:14.454951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a324c-08ea-461d-8b44-d1599b06306e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:14.457227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:14.477547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcb66d-8e62-4c2b-ae7e-0264a7608013 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:14.479055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcb66d-8e62-4c2b-ae7e-0264a7608013 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:14.490815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264151b-dad1-42d3-a592-e200c7576693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:14.491476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264151b-dad1-42d3-a592-e200c7576693 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:14.491759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264151b-dad1-42d3-a592-e200c7576693 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:14.492066 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264151b-dad1-42d3-a592-e200c7576693 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:14.492688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264151b-dad1-42d3-a592-e200c7576693 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:14.492997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264151b-dad1-42d3-a592-e200c7576693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:14.494381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264151b-dad1-42d3-a592-e200c7576693 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:14.499033 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 418/853] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:32:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:14.500743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264151b-dad1-42d3-a592-e200c7576693 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:14.584131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 436/854] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:32:14.585776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcb66d-8e62-4c2b-ae7e-0264a7608013 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:14.611480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264151b-dad1-42d3-a592-e200c7576693 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:14.651255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70efde-8b6e-4d9e-a719-8fc7c37d2105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:14.651913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70efde-8b6e-4d9e-a719-8fc7c37d2105 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:14.652204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70efde-8b6e-4d9e-a719-8fc7c37d2105 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:14.652485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70efde-8b6e-4d9e-a719-8fc7c37d2105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:14.662688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70efde-8b6e-4d9e-a719-8fc7c37d2105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:14.664095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70efde-8b6e-4d9e-a719-8fc7c37d2105 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:14.666609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:14.675657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-be70efde-8b6e-4d9e-a719-8fc7c37d2105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:14.679568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:14.680954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:14.681872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:14.681872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:14.681872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:14.681872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 419/855] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:14.681872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70efde-8b6e-4d9e-a719-8fc7c37d2105 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:14.694872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcb66d-8e62-4c2b-ae7e-0264a7608013 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:14.727823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bffd73-1ee4-42f9-ac9e-9a62f3c33d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:14.728450 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bffd73-1ee4-42f9-ac9e-9a62f3c33d9d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:14.728736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bffd73-1ee4-42f9-ac9e-9a62f3c33d9d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:14.729017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bffd73-1ee4-42f9-ac9e-9a62f3c33d9d None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:14.729646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bffd73-1ee4-42f9-ac9e-9a62f3c33d9d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:14.729952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bffd73-1ee4-42f9-ac9e-9a62f3c33d9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:14.731328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bffd73-1ee4-42f9-ac9e-9a62f3c33d9d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:14.737221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 437/856] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:32:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:14.738387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57508844-bb99-4bbd-b81e-be2dfdb60b73 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:14.738673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57508844-bb99-4bbd-b81e-be2dfdb60b73 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:14.738947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57508844-bb99-4bbd-b81e-be2dfdb60b73 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:14.740197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 438/857] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:14.740913 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57508844-bb99-4bbd-b81e-be2dfdb60b73 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:14.800852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70efde-8b6e-4d9e-a719-8fc7c37d2105 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:14.840340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d39bc-147a-452a-9101-baca3929a383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:14.841013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d39bc-147a-452a-9101-baca3929a383 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:14.841301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d39bc-147a-452a-9101-baca3929a383 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:14.841579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d39bc-147a-452a-9101-baca3929a383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:14.853414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d39bc-147a-452a-9101-baca3929a383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:14.854909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d39bc-147a-452a-9101-baca3929a383 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:14.857105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57508844-bb99-4bbd-b81e-be2dfdb60b73 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-f32d39bc-147a-452a-9101-baca3929a383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:14.870380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:14.871432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:14.872385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:14.872385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:14.872385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:14.872385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 420/858] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:14.873256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f4d9-24b4-4c39-96f9-be5e83f03f29 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:14.873547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f4d9-24b4-4c39-96f9-be5e83f03f29 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:14.873825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f4d9-24b4-4c39-96f9-be5e83f03f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:14.894350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-5d03f4d9-24b4-4c39-96f9-be5e83f03f29 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:14.895552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5d03f4d9-24b4-4c39-96f9-be5e83f03f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:14.902106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa85c57-d64a-4516-ac26-ec71a57a71e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:14.902723 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa85c57-d64a-4516-ac26-ec71a57a71e0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:14.903002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa85c57-d64a-4516-ac26-ec71a57a71e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:14.903280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa85c57-d64a-4516-ac26-ec71a57a71e0 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:14.903957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa85c57-d64a-4516-ac26-ec71a57a71e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:14.904505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa85c57-d64a-4516-ac26-ec71a57a71e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:14.905639 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa85c57-d64a-4516-ac26-ec71a57a71e0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:14.922134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 439/859] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:14.922134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa85c57-d64a-4516-ac26-ec71a57a71e0 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:14.926468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:14.949885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:14.974030 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 421/860] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:14.980787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03f4d9-24b4-4c39-96f9-be5e83f03f29 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:15.041969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa85c57-d64a-4516-ac26-ec71a57a71e0 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:15.089336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea0683-c0ae-416d-83eb-1b94d729319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:15.089950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea0683-c0ae-416d-83eb-1b94d729319f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:15.090239 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea0683-c0ae-416d-83eb-1b94d729319f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:15.090522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea0683-c0ae-416d-83eb-1b94d729319f None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:15.091172 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea0683-c0ae-416d-83eb-1b94d729319f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:15.091481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea0683-c0ae-416d-83eb-1b94d729319f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:15.092045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03f4d9-24b4-4c39-96f9-be5e83f03f29 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:15.092888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea0683-c0ae-416d-83eb-1b94d729319f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:15.109144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 440/861] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:15.114117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea0683-c0ae-416d-83eb-1b94d729319f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:15.136848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8238a-5fc6-4d8b-afc1-44430c5501a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:15.137830 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8238a-5fc6-4d8b-afc1-44430c5501a6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:15.138354 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8238a-5fc6-4d8b-afc1-44430c5501a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:15.138837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8238a-5fc6-4d8b-afc1-44430c5501a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:15.146861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8238a-5fc6-4d8b-afc1-44430c5501a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:15.149108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8238a-5fc6-4d8b-afc1-44430c5501a6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-63a8238a-5fc6-4d8b-afc1-44430c5501a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:15.161685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:15.162683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:15.163617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:15.163617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:15.163617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:15.163617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:15.163617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:15.163617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:15.163617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:15.163617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:15.170440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 422/862] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:15.172124 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8238a-5fc6-4d8b-afc1-44430c5501a6 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:15.191586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:15.218716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:15.244826 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea0683-c0ae-416d-83eb-1b94d729319f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:15.281130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8238a-5fc6-4d8b-afc1-44430c5501a6 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:15.282123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451608a6-9c9a-46b9-adb7-88ad7de5d24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:15.282793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451608a6-9c9a-46b9-adb7-88ad7de5d24f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:15.283080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451608a6-9c9a-46b9-adb7-88ad7de5d24f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:15.283360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451608a6-9c9a-46b9-adb7-88ad7de5d24f None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:15.290232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451608a6-9c9a-46b9-adb7-88ad7de5d24f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:15.291671 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451608a6-9c9a-46b9-adb7-88ad7de5d24f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-451608a6-9c9a-46b9-adb7-88ad7de5d24f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:15.304305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:15.305228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:15.306173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:15.306173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:15.306173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:15.306173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:15.306173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:15.306173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:15.306173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:15.306173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:15.306173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 441/863] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:15.306173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451608a6-9c9a-46b9-adb7-88ad7de5d24f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:15.324097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7c0d-de26-4fc2-a142-af332d5ccf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:15.324715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7c0d-de26-4fc2-a142-af332d5ccf1f None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:15.324975 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7c0d-de26-4fc2-a142-af332d5ccf1f None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:15.325250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7c0d-de26-4fc2-a142-af332d5ccf1f None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:15.325938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a7c0d-de26-4fc2-a142-af332d5ccf1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:15.326244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a7c0d-de26-4fc2-a142-af332d5ccf1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:15.327664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a7c0d-de26-4fc2-a142-af332d5ccf1f None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:15.339695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 423/864] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:15.340608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7c0d-de26-4fc2-a142-af332d5ccf1f None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:15.416900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451608a6-9c9a-46b9-adb7-88ad7de5d24f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:15.442643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:15.444032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:15.446649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7c0d-de26-4fc2-a142-af332d5ccf1f None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:15.454612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5860d-67c2-4780-bf42-a473548121ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:15.455241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5860d-67c2-4780-bf42-a473548121ab None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:15.455533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5860d-67c2-4780-bf42-a473548121ab None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:15.455809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5860d-67c2-4780-bf42-a473548121ab None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:15.456422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5860d-67c2-4780-bf42-a473548121ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:15.457915 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5860d-67c2-4780-bf42-a473548121ab None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-2bb5860d-67c2-4780-bf42-a473548121ab None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:15.466874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:15.467803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:15.468751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 442/865] 10.222.0.98 () {56 vars in 1132 bytes} [Fri Aug 26 15:32:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:15.469939 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5860d-67c2-4780-bf42-a473548121ab None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:15.484631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634ef3-a39b-4ecf-b5c3-0e788082a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:15.485301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634ef3-a39b-4ecf-b5c3-0e788082a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:15.486049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634ef3-a39b-4ecf-b5c3-0e788082a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:15.486324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634ef3-a39b-4ecf-b5c3-0e788082a0f5 None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:15.487017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62634ef3-a39b-4ecf-b5c3-0e788082a0f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:15.487378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62634ef3-a39b-4ecf-b5c3-0e788082a0f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:15.488873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62634ef3-a39b-4ecf-b5c3-0e788082a0f5 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:15.503129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 424/866] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:15.503891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634ef3-a39b-4ecf-b5c3-0e788082a0f5 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:15.583381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5860d-67c2-4780-bf42-a473548121ab None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:15.608675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634ef3-a39b-4ecf-b5c3-0e788082a0f5 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:15.621173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cf067-144f-4edc-9661-a41f880da9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:15.621778 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf067-144f-4edc-9661-a41f880da9d4 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:15.622125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf067-144f-4edc-9661-a41f880da9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:15.622410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf067-144f-4edc-9661-a41f880da9d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:15.623136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cf067-144f-4edc-9661-a41f880da9d4 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=1fae47d6b7a7472a9d94636bb126f802) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:15.623446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cf067-144f-4edc-9661-a41f880da9d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:15.624905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cf067-144f-4edc-9661-a41f880da9d4 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:15.633712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 443/867] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:15 2022] GET /identity/v3/role_assignments?user.id=1fae47d6b7a7472a9d94636bb126f802&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=6ba5a5add2174983be37258aad1ee404 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:15.635974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cf067-144f-4edc-9661-a41f880da9d4 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:15.653180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4a53c-ff98-46f0-8554-f9789de88606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:15.654221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4a53c-ff98-46f0-8554-f9789de88606 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:15.654796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4a53c-ff98-46f0-8554-f9789de88606 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:15.655281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4a53c-ff98-46f0-8554-f9789de88606 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:15.656348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4a53c-ff98-46f0-8554-f9789de88606 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:15.656919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4a53c-ff98-46f0-8554-f9789de88606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:15.659162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4a53c-ff98-46f0-8554-f9789de88606 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:15.665245 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 425/868] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Aug 26 15:32:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:15.686729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4a53c-ff98-46f0-8554-f9789de88606 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:15.701605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:15.726121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:15.768849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cf067-144f-4edc-9661-a41f880da9d4 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:15.817907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd07eaa-30ee-4cb2-8e2c-2edf51d17f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:15.818534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd07eaa-30ee-4cb2-8e2c-2edf51d17f18 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:15.818818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd07eaa-30ee-4cb2-8e2c-2edf51d17f18 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:15.819093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd07eaa-30ee-4cb2-8e2c-2edf51d17f18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:15.819840 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd07eaa-30ee-4cb2-8e2c-2edf51d17f18 None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=b24a6d0c1dbe49a18b3b8b6732a938be) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:15.820295 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd07eaa-30ee-4cb2-8e2c-2edf51d17f18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:15.822765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd07eaa-30ee-4cb2-8e2c-2edf51d17f18 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:15.826458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4a53c-ff98-46f0-8554-f9789de88606 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:15.832294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 444/869] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:15 2022] GET /identity/v3/role_assignments?user.id=b24a6d0c1dbe49a18b3b8b6732a938be&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:15.870555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a25f3-9b0d-4b8c-af3c-31d3350e84e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:15.871170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a25f3-9b0d-4b8c-af3c-31d3350e84e1 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:15.871454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a25f3-9b0d-4b8c-af3c-31d3350e84e1 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:15.871730 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a25f3-9b0d-4b8c-af3c-31d3350e84e1 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:15.873181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a25f3-9b0d-4b8c-af3c-31d3350e84e1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:15.873181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a25f3-9b0d-4b8c-af3c-31d3350e84e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:15.874069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a25f3-9b0d-4b8c-af3c-31d3350e84e1 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:15.880726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 426/870] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Aug 26 15:32:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:16.660145 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97adf11-e5aa-4020-90a0-5da04d5b918b None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:16.660662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97adf11-e5aa-4020-90a0-5da04d5b918b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:16.660752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97adf11-e5aa-4020-90a0-5da04d5b918b None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:16.662228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 445/871] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:16.667459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad976-43bd-40fb-a8be-6bb0a0cb4c35 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:16.667852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad976-43bd-40fb-a8be-6bb0a0cb4c35 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:16.668043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad976-43bd-40fb-a8be-6bb0a0cb4c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:16.687563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-dfbad976-43bd-40fb-a8be-6bb0a0cb4c35 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:16.688949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-dfbad976-43bd-40fb-a8be-6bb0a0cb4c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:16.758871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 427/872] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:17.024432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97adf11-e5aa-4020-90a0-5da04d5b918b None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:17.136766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97adf11-e5aa-4020-90a0-5da04d5b918b None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:17.154701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:17.173882 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065cbe0-18fd-413b-80f9-2eb43737e61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:17.174838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065cbe0-18fd-413b-80f9-2eb43737e61c None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:17.175341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065cbe0-18fd-413b-80f9-2eb43737e61c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:17.175825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065cbe0-18fd-413b-80f9-2eb43737e61c None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:17.176701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065cbe0-18fd-413b-80f9-2eb43737e61c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:17.178280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065cbe0-18fd-413b-80f9-2eb43737e61c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:17.249135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 446/873] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:32:17 2022] POST /identity/v3/services => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:17.779992 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc18ee8-cf9c-476d-a281-126be649a32e None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:17.781019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc18ee8-cf9c-476d-a281-126be649a32e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:17.781513 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc18ee8-cf9c-476d-a281-126be649a32e None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:17.783424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 428/874] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:17.789023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32b3b5-ec46-4659-886b-a3805e8576e6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:17.789863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32b3b5-ec46-4659-886b-a3805e8576e6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:17.790351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32b3b5-ec46-4659-886b-a3805e8576e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:17.811093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ab32b3b5-ec46-4659-886b-a3805e8576e6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:17.812671 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ab32b3b5-ec46-4659-886b-a3805e8576e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:17.842258 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82057be8-c166-4297-9769-dcf5cc153dbe None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:17.842518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82057be8-c166-4297-9769-dcf5cc153dbe None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:17.842805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82057be8-c166-4297-9769-dcf5cc153dbe None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:17.844409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 429/875] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:17.849410 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e70aa-46eb-47d7-8b5e-c76c437b3745 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:17.849690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e70aa-46eb-47d7-8b5e-c76c437b3745 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:17.849973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e70aa-46eb-47d7-8b5e-c76c437b3745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:17.869161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-145e70aa-46eb-47d7-8b5e-c76c437b3745 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:17.870296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-145e70aa-46eb-47d7-8b5e-c76c437b3745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:17.892290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:17.923907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 447/876] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:17.923907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f575b-0ecb-44c2-a6cb-a6c192e9e77d None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:17.923907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f575b-0ecb-44c2-a6cb-a6c192e9e77d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:17.923907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f575b-0ecb-44c2-a6cb-a6c192e9e77d None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:17.926541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 448/877] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:17.928110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce802491-8e86-4027-9e50-3ad93aa8b24c None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:17.928407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce802491-8e86-4027-9e50-3ad93aa8b24c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:17.929031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce802491-8e86-4027-9e50-3ad93aa8b24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:17.948169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ce802491-8e86-4027-9e50-3ad93aa8b24c None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:17.949384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ce802491-8e86-4027-9e50-3ad93aa8b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:17.949894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 430/878] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:18.022743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 449/879] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:18.191761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e70aa-46eb-47d7-8b5e-c76c437b3745 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:18.217886 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce802491-8e86-4027-9e50-3ad93aa8b24c None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:18.257875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:18.306386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e70aa-46eb-47d7-8b5e-c76c437b3745 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:18.343714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3a8ee-0f8a-48b8-aa7a-74dc97fb36ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:18.344105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce802491-8e86-4027-9e50-3ad93aa8b24c None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:18.344428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3a8ee-0f8a-48b8-aa7a-74dc97fb36ea None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:18.345268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3a8ee-0f8a-48b8-aa7a-74dc97fb36ea None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:18.345542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3a8ee-0f8a-48b8-aa7a-74dc97fb36ea None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:18.346262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3a8ee-0f8a-48b8-aa7a-74dc97fb36ea None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:18.347824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3a8ee-0f8a-48b8-aa7a-74dc97fb36ea None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-eab3a8ee-0f8a-48b8-aa7a-74dc97fb36ea None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:18.357624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:18.358729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:18.359634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 431/880] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Aug 26 15:32:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:18.360556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3a8ee-0f8a-48b8-aa7a-74dc97fb36ea None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:18.373291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:18.388158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7876741-583c-400d-9abc-3d2e8a4b366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:18.388774 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876741-583c-400d-9abc-3d2e8a4b366c None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:18.389060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876741-583c-400d-9abc-3d2e8a4b366c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:18.389335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876741-583c-400d-9abc-3d2e8a4b366c None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:18.389989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7876741-583c-400d-9abc-3d2e8a4b366c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:18.391918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7876741-583c-400d-9abc-3d2e8a4b366c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:18.459549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 450/881] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:32:18 2022] POST /identity/v3/services => generated 269 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:18.460501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7876741-583c-400d-9abc-3d2e8a4b366c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:18.472025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3a8ee-0f8a-48b8-aa7a-74dc97fb36ea None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:18.537206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4b958-b239-48aa-a956-4d1ebab5e308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:18.537859 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4b958-b239-48aa-a956-4d1ebab5e308 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:18.538145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4b958-b239-48aa-a956-4d1ebab5e308 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:18.538692 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4b958-b239-48aa-a956-4d1ebab5e308 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:18.555103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4b958-b239-48aa-a956-4d1ebab5e308 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-79d4b958-b239-48aa-a956-4d1ebab5e308 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:18.561796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:18.562815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:18.563720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:18.564430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 432/882] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Aug 26 15:32:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:18.571199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4b958-b239-48aa-a956-4d1ebab5e308 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:18.590506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:18.612998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7876741-583c-400d-9abc-3d2e8a4b366c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:18.665077 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20daebd-1ba7-4b2b-ba11-b977bb27fd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:18.665721 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20daebd-1ba7-4b2b-ba11-b977bb27fd58 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:18.666010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20daebd-1ba7-4b2b-ba11-b977bb27fd58 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:18.666289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20daebd-1ba7-4b2b-ba11-b977bb27fd58 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:18.666971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20daebd-1ba7-4b2b-ba11-b977bb27fd58 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:18.667283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20daebd-1ba7-4b2b-ba11-b977bb27fd58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:18.668728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20daebd-1ba7-4b2b-ba11-b977bb27fd58 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:18.669217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:18.678292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 451/883] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Aug 26 15:32:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:18.680012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20daebd-1ba7-4b2b-ba11-b977bb27fd58 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:18.714390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4b958-b239-48aa-a956-4d1ebab5e308 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:18.749480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9db4cd-7f39-4038-b137-5c40e737d654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:18.750460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9db4cd-7f39-4038-b137-5c40e737d654 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:18.751490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9db4cd-7f39-4038-b137-5c40e737d654 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:18.752002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9db4cd-7f39-4038-b137-5c40e737d654 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:18.752947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9db4cd-7f39-4038-b137-5c40e737d654 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:18.753465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9db4cd-7f39-4038-b137-5c40e737d654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:18.755058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9db4cd-7f39-4038-b137-5c40e737d654 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:18.764115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 433/884] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Aug 26 15:32:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:18.770669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9db4cd-7f39-4038-b137-5c40e737d654 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:18.799666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20daebd-1ba7-4b2b-ba11-b977bb27fd58 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:18.847225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5883a-a9fc-48f1-aa5d-ac5f74bca434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:18.848126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5883a-a9fc-48f1-aa5d-ac5f74bca434 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:18.848418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5883a-a9fc-48f1-aa5d-ac5f74bca434 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:18.849569 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5883a-a9fc-48f1-aa5d-ac5f74bca434 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:18.850223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5883a-a9fc-48f1-aa5d-ac5f74bca434 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:18.850534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5883a-a9fc-48f1-aa5d-ac5f74bca434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:18.851927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5883a-a9fc-48f1-aa5d-ac5f74bca434 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:18.859982 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 452/885] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Aug 26 15:32:18 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:18.890077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9db4cd-7f39-4038-b137-5c40e737d654 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:18.919840 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:18.929087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d67a01-a4fc-4c47-a22e-4f482d7a99f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:18.929732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d67a01-a4fc-4c47-a22e-4f482d7a99f0 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:18.930111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d67a01-a4fc-4c47-a22e-4f482d7a99f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:18.930386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d67a01-a4fc-4c47-a22e-4f482d7a99f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:18.942841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d67a01-a4fc-4c47-a22e-4f482d7a99f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:18.944509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d67a01-a4fc-4c47-a22e-4f482d7a99f0 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-b2d67a01-a4fc-4c47-a22e-4f482d7a99f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:18.961289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:18.962316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:18.963316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:18.963316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:18.963316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:18.969855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 434/886] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:18.975242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5883a-a9fc-48f1-aa5d-ac5f74bca434 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:19.005904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:19.095518 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5883a-a9fc-48f1-aa5d-ac5f74bca434 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:19.132394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370bbe7-14bc-4e47-ace5-976da5d1a4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:19.133345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370bbe7-14bc-4e47-ace5-976da5d1a4a7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:19.133852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370bbe7-14bc-4e47-ace5-976da5d1a4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:19.134351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370bbe7-14bc-4e47-ace5-976da5d1a4a7 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:19.135193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370bbe7-14bc-4e47-ace5-976da5d1a4a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:19.135696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370bbe7-14bc-4e47-ace5-976da5d1a4a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:19.137355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370bbe7-14bc-4e47-ace5-976da5d1a4a7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:19.153529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 453/887] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:19.160461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d67a01-a4fc-4c47-a22e-4f482d7a99f0 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:19.191950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2df18-49f6-4523-99fb-7f48907c0a7b None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:19.192223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2df18-49f6-4523-99fb-7f48907c0a7b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:19.192541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2df18-49f6-4523-99fb-7f48907c0a7b None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:19.194165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 454/888] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:19.199263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb386284-0030-46c7-a6a2-7d2421c36a88 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:19.199571 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb386284-0030-46c7-a6a2-7d2421c36a88 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:19.200012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb386284-0030-46c7-a6a2-7d2421c36a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:19.218815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-cb386284-0030-46c7-a6a2-7d2421c36a88 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:19.220349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cb386284-0030-46c7-a6a2-7d2421c36a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:19.287942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d67a01-a4fc-4c47-a22e-4f482d7a99f0 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:19.293299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 455/889] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:19.325182 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225704-a4c4-4401-89db-0428c5cc6e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:19.326326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225704-a4c4-4401-89db-0428c5cc6e59 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:19.326822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225704-a4c4-4401-89db-0428c5cc6e59 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:19.327316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225704-a4c4-4401-89db-0428c5cc6e59 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:19.335447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf225704-a4c4-4401-89db-0428c5cc6e59 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:19.337148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf225704-a4c4-4401-89db-0428c5cc6e59 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-bf225704-a4c4-4401-89db-0428c5cc6e59 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:19.348657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:19.349728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:19.350776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:19.350776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:19.350776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:19.350776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:19.350776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:19.350776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:19.350776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:19.350776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:19.350776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:19.350776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:19.353316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 435/890] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Aug 26 15:32:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:19.358600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb386284-0030-46c7-a6a2-7d2421c36a88 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:19.372112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:19.468406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb386284-0030-46c7-a6a2-7d2421c36a88 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:19.506463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52c33a-6a15-4462-b97d-cd58c8aba52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:19.507437 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52c33a-6a15-4462-b97d-cd58c8aba52d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:19.507975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52c33a-6a15-4462-b97d-cd58c8aba52d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:19.508502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52c33a-6a15-4462-b97d-cd58c8aba52d None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:19.509493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52c33a-6a15-4462-b97d-cd58c8aba52d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:19.510009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52c33a-6a15-4462-b97d-cd58c8aba52d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:19.511640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52c33a-6a15-4462-b97d-cd58c8aba52d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:19.530106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 456/891] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Aug 26 15:32:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:19.536967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225704-a4c4-4401-89db-0428c5cc6e59 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:19.573242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52c33a-6a15-4462-b97d-cd58c8aba52d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:19.581619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:19.660716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225704-a4c4-4401-89db-0428c5cc6e59 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:19.661002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:19.704069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e009f-2793-4a7c-9657-238271ef61a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:19.704628 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52c33a-6a15-4462-b97d-cd58c8aba52d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:19.705232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e009f-2793-4a7c-9657-238271ef61a9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:19.705705 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e009f-2793-4a7c-9657-238271ef61a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:19.706185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e009f-2793-4a7c-9657-238271ef61a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:19.748964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b0c49-85cf-4a67-b895-230856cdc4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:19.749675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b0c49-85cf-4a67-b895-230856cdc4dd None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:19.750022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b0c49-85cf-4a67-b895-230856cdc4dd None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:19.750365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b0c49-85cf-4a67-b895-230856cdc4dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:19.751046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b0c49-85cf-4a67-b895-230856cdc4dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:19.753024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b0c49-85cf-4a67-b895-230856cdc4dd None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-df7b0c49-85cf-4a67-b895-230856cdc4dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:19.764562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:19.765633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:19.772045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 457/892] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:32:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:19.774429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b0c49-85cf-4a67-b895-230856cdc4dd None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:19.788279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e009f-2793-4a7c-9657-238271ef61a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:19.790187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e009f-2793-4a7c-9657-238271ef61a9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:19.835431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:19.899034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b0c49-85cf-4a67-b895-230856cdc4dd None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:19.911968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 436/893] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:19 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:32:19.916012 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e009f-2793-4a7c-9657-238271ef61a9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:19.940888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:19.947415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad127b-a7bf-4edd-8382-2bcafb890380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:19.948061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad127b-a7bf-4edd-8382-2bcafb890380 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:19.948345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad127b-a7bf-4edd-8382-2bcafb890380 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:19.948622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad127b-a7bf-4edd-8382-2bcafb890380 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:19.949276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad127b-a7bf-4edd-8382-2bcafb890380 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:19.949707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad127b-a7bf-4edd-8382-2bcafb890380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:19.951661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad127b-a7bf-4edd-8382-2bcafb890380 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:19.958574 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 458/894] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:32:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:19.965098 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad127b-a7bf-4edd-8382-2bcafb890380 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:20.041254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e009f-2793-4a7c-9657-238271ef61a9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:20.085645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef7cc0-1b3a-42a8-bf1b-200d220e1c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:20.086363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef7cc0-1b3a-42a8-bf1b-200d220e1c45 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.086653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef7cc0-1b3a-42a8-bf1b-200d220e1c45 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.087180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef7cc0-1b3a-42a8-bf1b-200d220e1c45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.090889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad127b-a7bf-4edd-8382-2bcafb890380 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:20.096602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:20.099177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef7cc0-1b3a-42a8-bf1b-200d220e1c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:20.100664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef7cc0-1b3a-42a8-bf1b-200d220e1c45 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-5bef7cc0-1b3a-42a8-bf1b-200d220e1c45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:20.117515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:20.118531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:20.119447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:20.119447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:20.119447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:20.123395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 437/895] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:20.124703 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef7cc0-1b3a-42a8-bf1b-200d220e1c45 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:20.136982 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45098978-84de-4846-8744-b6db627a89c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:20.138342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45098978-84de-4846-8744-b6db627a89c4 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.139235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45098978-84de-4846-8744-b6db627a89c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.139735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45098978-84de-4846-8744-b6db627a89c4 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.140839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45098978-84de-4846-8744-b6db627a89c4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:20.141384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45098978-84de-4846-8744-b6db627a89c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:20.143899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45098978-84de-4846-8744-b6db627a89c4 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:20.150699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 459/896] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:32:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:20.156315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45098978-84de-4846-8744-b6db627a89c4 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:20.237537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:20.238496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef7cc0-1b3a-42a8-bf1b-200d220e1c45 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:20.277995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3929e7a-5d47-479a-96ef-8d92b18c1301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:20.279167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929e7a-5d47-479a-96ef-8d92b18c1301 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.280052 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929e7a-5d47-479a-96ef-8d92b18c1301 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.280527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929e7a-5d47-479a-96ef-8d92b18c1301 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.282106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3929e7a-5d47-479a-96ef-8d92b18c1301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:20.282622 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3929e7a-5d47-479a-96ef-8d92b18c1301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:20.284748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3929e7a-5d47-479a-96ef-8d92b18c1301 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:20.297038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45098978-84de-4846-8744-b6db627a89c4 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:20.305879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 438/897] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:20.307492 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3929e7a-5d47-479a-96ef-8d92b18c1301 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:20.340819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f5696-1d0a-43bd-a6e5-368e526c7ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:20.341437 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f5696-1d0a-43bd-a6e5-368e526c7ff9 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.341709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f5696-1d0a-43bd-a6e5-368e526c7ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.342035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f5696-1d0a-43bd-a6e5-368e526c7ff9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.342748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f5696-1d0a-43bd-a6e5-368e526c7ff9 None admin] RBAC: Adding query filter params (interface=public, service_id=f87c9403085b4fe589078b3fc88ac458, region_id=RegionOne) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:20.343075 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f5696-1d0a-43bd-a6e5-368e526c7ff9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:20.345004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f5696-1d0a-43bd-a6e5-368e526c7ff9 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:20.354898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 460/898] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Aug 26 15:32:20 2022] GET /identity/v3/endpoints?service_id=f87c9403085b4fe589078b3fc88ac458&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:20.356214 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f5696-1d0a-43bd-a6e5-368e526c7ff9 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:20.372051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:20.424276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3929e7a-5d47-479a-96ef-8d92b18c1301 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:20.462720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95f6c9-8b5e-4bb3-8884-92ceb5aef4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:20.463432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95f6c9-8b5e-4bb3-8884-92ceb5aef4e6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.463917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95f6c9-8b5e-4bb3-8884-92ceb5aef4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.464271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95f6c9-8b5e-4bb3-8884-92ceb5aef4e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.473475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95f6c9-8b5e-4bb3-8884-92ceb5aef4e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:20.475347 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95f6c9-8b5e-4bb3-8884-92ceb5aef4e6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-5a95f6c9-8b5e-4bb3-8884-92ceb5aef4e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:20.487564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:20.488564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:20.489480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:20.489480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:20.489480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:20.489480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:20.489480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:20.489480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:20.489480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:20.489480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:20.489951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f5696-1d0a-43bd-a6e5-368e526c7ff9 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:20.492046 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 439/899] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:20.493340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf8cf-4dbc-45ab-89fa-7bae89c5ec00 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.493833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf8cf-4dbc-45ab-89fa-7bae89c5ec00 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.493945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf8cf-4dbc-45ab-89fa-7bae89c5ec00 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.495156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 440/900] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:20.495839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcf8cf-4dbc-45ab-89fa-7bae89c5ec00 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:20.520321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:20.531193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d3fc3-99b8-4007-af97-3f3db7bebc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:20.532404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d3fc3-99b8-4007-af97-3f3db7bebc9b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.532691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d3fc3-99b8-4007-af97-3f3db7bebc9b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.534607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d3fc3-99b8-4007-af97-3f3db7bebc9b None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.534607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d3fc3-99b8-4007-af97-3f3db7bebc9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:20.534918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d3fc3-99b8-4007-af97-3f3db7bebc9b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:20.548252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 461/901] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Aug 26 15:32:20 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:32:20.548252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a383fd-1a40-42e2-8d3d-c5dab9e694ad None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.548252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a383fd-1a40-42e2-8d3d-c5dab9e694ad None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.548252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a383fd-1a40-42e2-8d3d-c5dab9e694ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.565929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f8a383fd-1a40-42e2-8d3d-c5dab9e694ad None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:20.567262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f8a383fd-1a40-42e2-8d3d-c5dab9e694ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:20.664046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 462/902] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:20.681347 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcf8cf-4dbc-45ab-89fa-7bae89c5ec00 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:20.720845 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deba83f-2cb0-4cd7-9cca-1de28289fd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:20.721463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deba83f-2cb0-4cd7-9cca-1de28289fd97 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.721797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deba83f-2cb0-4cd7-9cca-1de28289fd97 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.722081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deba83f-2cb0-4cd7-9cca-1de28289fd97 None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.722845 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deba83f-2cb0-4cd7-9cca-1de28289fd97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:20.723162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deba83f-2cb0-4cd7-9cca-1de28289fd97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:20.724669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deba83f-2cb0-4cd7-9cca-1de28289fd97 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:20.732258 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:20.737697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 441/903] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:20.744372 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a383fd-1a40-42e2-8d3d-c5dab9e694ad None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:20.851414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a383fd-1a40-42e2-8d3d-c5dab9e694ad None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:20.869898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:20.888293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9711cc5-fe31-4484-8d6d-275144f78e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:20.888900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9711cc5-fe31-4484-8d6d-275144f78e2a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.889185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9711cc5-fe31-4484-8d6d-275144f78e2a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.889469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9711cc5-fe31-4484-8d6d-275144f78e2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.890279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9711cc5-fe31-4484-8d6d-275144f78e2a None admin] RBAC: Adding query filter params (role.id=3fec161072094f87aad16699524908bc, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=b24a6d0c1dbe49a18b3b8b6732a938be) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:20.890591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9711cc5-fe31-4484-8d6d-275144f78e2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:20.892096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9711cc5-fe31-4484-8d6d-275144f78e2a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:20.898699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 463/904] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:20 2022] GET /identity/v3/role_assignments?user.id=b24a6d0c1dbe49a18b3b8b6732a938be&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=3fec161072094f87aad16699524908bc => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:20.966798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e176b7-211c-478f-878b-f75ba38add26 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.967353 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e176b7-211c-478f-878b-f75ba38add26 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.967353 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e176b7-211c-478f-878b-f75ba38add26 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.970328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 442/905] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:20.973612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9711cc5-fe31-4484-8d6d-275144f78e2a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:20.976287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8598b0-03ce-44b7-99e1-0f5023c2f694 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:20.976576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8598b0-03ce-44b7-99e1-0f5023c2f694 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:20.976864 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8598b0-03ce-44b7-99e1-0f5023c2f694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:20.997462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-6a8598b0-03ce-44b7-99e1-0f5023c2f694 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:20.998872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6a8598b0-03ce-44b7-99e1-0f5023c2f694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:21.087761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9711cc5-fe31-4484-8d6d-275144f78e2a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:21.092004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 443/906] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:21.126480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eb4ce-0634-4a7c-b107-fb27b21e4206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:21.127103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eb4ce-0634-4a7c-b107-fb27b21e4206 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:21.127387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eb4ce-0634-4a7c-b107-fb27b21e4206 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:21.127663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eb4ce-0634-4a7c-b107-fb27b21e4206 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:21.128280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42eb4ce-0634-4a7c-b107-fb27b21e4206 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:21.129653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42eb4ce-0634-4a7c-b107-fb27b21e4206 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-a42eb4ce-0634-4a7c-b107-fb27b21e4206 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:21.138734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:21.139678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:21.140598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 464/907] 10.222.0.98 () {56 vars in 1132 bytes} [Fri Aug 26 15:32:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:21.145994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8598b0-03ce-44b7-99e1-0f5023c2f694 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:21.254472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8598b0-03ce-44b7-99e1-0f5023c2f694 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:21.256427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:21.288676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17111ea8-c8fd-45f8-b38e-d10ff1800eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:21.289287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17111ea8-c8fd-45f8-b38e-d10ff1800eb1 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:21.289568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17111ea8-c8fd-45f8-b38e-d10ff1800eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:21.289960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17111ea8-c8fd-45f8-b38e-d10ff1800eb1 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:21.290505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17111ea8-c8fd-45f8-b38e-d10ff1800eb1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:21.291195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17111ea8-c8fd-45f8-b38e-d10ff1800eb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:21.292603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17111ea8-c8fd-45f8-b38e-d10ff1800eb1 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:21.302881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 444/908] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Aug 26 15:32:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:21.304017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eb4ce-0634-4a7c-b107-fb27b21e4206 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:21.354378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:21.385245 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17111ea8-c8fd-45f8-b38e-d10ff1800eb1 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:21.412727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eb4ce-0634-4a7c-b107-fb27b21e4206 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:21.457355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cc2ca-855f-416d-96cf-c4a9b1b12449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:21.458116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cc2ca-855f-416d-96cf-c4a9b1b12449 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:21.458521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cc2ca-855f-416d-96cf-c4a9b1b12449 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:21.458801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cc2ca-855f-416d-96cf-c4a9b1b12449 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:21.459445 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cc2ca-855f-416d-96cf-c4a9b1b12449 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:21.460082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cc2ca-855f-416d-96cf-c4a9b1b12449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:21.462349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cc2ca-855f-416d-96cf-c4a9b1b12449 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:21.468881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 465/909] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Aug 26 15:32:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:21.475361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cc2ca-855f-416d-96cf-c4a9b1b12449 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:21.503201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17111ea8-c8fd-45f8-b38e-d10ff1800eb1 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:21.551678 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b2d06-8a77-46a8-a3f8-3ef96f552af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:21.552862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2d06-8a77-46a8-a3f8-3ef96f552af2 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:21.553145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2d06-8a77-46a8-a3f8-3ef96f552af2 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:21.553567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2d06-8a77-46a8-a3f8-3ef96f552af2 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:21.554335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b2d06-8a77-46a8-a3f8-3ef96f552af2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:21.556873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b2d06-8a77-46a8-a3f8-3ef96f552af2 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:21.590564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cc2ca-855f-416d-96cf-c4a9b1b12449 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:21.603131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:21.627549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0bf29-51b0-4e94-871e-d2aa617b7a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:21.628181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0bf29-51b0-4e94-871e-d2aa617b7a0a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:21.628461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0bf29-51b0-4e94-871e-d2aa617b7a0a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:21.628885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0bf29-51b0-4e94-871e-d2aa617b7a0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:21.629618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0bf29-51b0-4e94-871e-d2aa617b7a0a None admin] RBAC: Adding query filter params (interface=public, service_id=00d79ea4c9614959861ea5d4c947b1cf, region_id=RegionOne) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:21.629943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0bf29-51b0-4e94-871e-d2aa617b7a0a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:21.631358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0bf29-51b0-4e94-871e-d2aa617b7a0a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:21.648954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 466/910] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Aug 26 15:32:21 2022] GET /identity/v3/endpoints?service_id=00d79ea4c9614959861ea5d4c947b1cf&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:21.648954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0bf29-51b0-4e94-871e-d2aa617b7a0a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:21.649210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 445/911] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:32:21 2022] POST /identity/v3/services => generated 241 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:32:21.761388 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0bf29-51b0-4e94-871e-d2aa617b7a0a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:21.804070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34745a-9320-48d4-b578-cca8980745f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:21.804676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34745a-9320-48d4-b578-cca8980745f4 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:21.804958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34745a-9320-48d4-b578-cca8980745f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:21.805232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34745a-9320-48d4-b578-cca8980745f4 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:21.805831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34745a-9320-48d4-b578-cca8980745f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:21.807186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34745a-9320-48d4-b578-cca8980745f4 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:21.814185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 467/912] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Aug 26 15:32:21 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:32:22.543503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb328d-7e7c-4e96-90d3-5066cf331c45 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:22.544050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb328d-7e7c-4e96-90d3-5066cf331c45 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:22.544171 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb328d-7e7c-4e96-90d3-5066cf331c45 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:22.545593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 446/913] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:22.551720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c64d9e-3347-4d35-99d0-c03115068e9e None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:22.552575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c64d9e-3347-4d35-99d0-c03115068e9e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:22.553062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c64d9e-3347-4d35-99d0-c03115068e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:22.571374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-46c64d9e-3347-4d35-99d0-c03115068e9e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:22.572914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-46c64d9e-3347-4d35-99d0-c03115068e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:22.608789 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:22.653872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 468/914] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:22.921149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490084a-25fb-407b-b70a-527b03164e25 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:22.921343 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490084a-25fb-407b-b70a-527b03164e25 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:22.921628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490084a-25fb-407b-b70a-527b03164e25 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:22.923501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 447/915] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:22.928955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be87d0-48ae-45aa-9340-08c76bb903e1 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:22.929252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490084a-25fb-407b-b70a-527b03164e25 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:22.929370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be87d0-48ae-45aa-9340-08c76bb903e1 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:22.929539 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be87d0-48ae-45aa-9340-08c76bb903e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:22.947881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-60be87d0-48ae-45aa-9340-08c76bb903e1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:22.949209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-60be87d0-48ae-45aa-9340-08c76bb903e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:22.959047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:23.034765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 469/916] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:23.055739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490084a-25fb-407b-b70a-527b03164e25 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:23.091013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe5807-99a3-499f-b62b-72b67ebf1422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:23.091646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe5807-99a3-499f-b62b-72b67ebf1422 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:23.091961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe5807-99a3-499f-b62b-72b67ebf1422 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:23.092244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe5807-99a3-499f-b62b-72b67ebf1422 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:23.092830 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe5807-99a3-499f-b62b-72b67ebf1422 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:23.094754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe5807-99a3-499f-b62b-72b67ebf1422 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-9ffe5807-99a3-499f-b62b-72b67ebf1422 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:23.103798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:23.104905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:23.106199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 448/917] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:32:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:23.110955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be87d0-48ae-45aa-9340-08c76bb903e1 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:23.214424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:23.221398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be87d0-48ae-45aa-9340-08c76bb903e1 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:23.256514 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fcc63-4272-4843-8fa6-a203a748b349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:23.257129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fcc63-4272-4843-8fa6-a203a748b349 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:23.257418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fcc63-4272-4843-8fa6-a203a748b349 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:23.257699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fcc63-4272-4843-8fa6-a203a748b349 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:23.258340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4fcc63-4272-4843-8fa6-a203a748b349 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:23.258650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4fcc63-4272-4843-8fa6-a203a748b349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:23.260112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4fcc63-4272-4843-8fa6-a203a748b349 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:23.266492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 470/918] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:32:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:23.275911 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe5807-99a3-499f-b62b-72b67ebf1422 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:23.329263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fcc63-4272-4843-8fa6-a203a748b349 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:23.383795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:23.396511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe5807-99a3-499f-b62b-72b67ebf1422 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:23.434612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a83295-9347-4c0a-a74e-f221a1d2ca86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:23.435610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a83295-9347-4c0a-a74e-f221a1d2ca86 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:23.436167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a83295-9347-4c0a-a74e-f221a1d2ca86 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:23.437076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a83295-9347-4c0a-a74e-f221a1d2ca86 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:23.438623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a83295-9347-4c0a-a74e-f221a1d2ca86 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:23.439199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a83295-9347-4c0a-a74e-f221a1d2ca86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:23.441301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a83295-9347-4c0a-a74e-f221a1d2ca86 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:23.448553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 449/919] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Aug 26 15:32:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:23.453592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a83295-9347-4c0a-a74e-f221a1d2ca86 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:23.457396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fcc63-4272-4843-8fa6-a203a748b349 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:23.497155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2f61a-3d30-4834-a548-0d71817bb69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:23.497708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2f61a-3d30-4834-a548-0d71817bb69b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:23.497991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2f61a-3d30-4834-a548-0d71817bb69b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:23.498268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2f61a-3d30-4834-a548-0d71817bb69b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:23.512515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2f61a-3d30-4834-a548-0d71817bb69b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-9da2f61a-3d30-4834-a548-0d71817bb69b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:23.517643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:23.518618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:23.519503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:23.523785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 471/920] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Aug 26 15:32:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:23.524783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2f61a-3d30-4834-a548-0d71817bb69b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:23.546139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:23.589100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a83295-9347-4c0a-a74e-f221a1d2ca86 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:23.632146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b8ec0-457d-44af-a6dc-abe5e4f51f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:23.633143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b8ec0-457d-44af-a6dc-abe5e4f51f88 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:23.633659 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b8ec0-457d-44af-a6dc-abe5e4f51f88 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:23.634151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b8ec0-457d-44af-a6dc-abe5e4f51f88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:23.635044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b8ec0-457d-44af-a6dc-abe5e4f51f88 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:23.636641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b8ec0-457d-44af-a6dc-abe5e4f51f88 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:23.647797 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2f61a-3d30-4834-a548-0d71817bb69b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:23.681711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446bcd0-a377-4465-bbc8-0f828e36ff4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:23.682310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446bcd0-a377-4465-bbc8-0f828e36ff4a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:23.682591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446bcd0-a377-4465-bbc8-0f828e36ff4a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:23.682866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446bcd0-a377-4465-bbc8-0f828e36ff4a None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:23.683523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446bcd0-a377-4465-bbc8-0f828e36ff4a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:23.683820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446bcd0-a377-4465-bbc8-0f828e36ff4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:23.687064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446bcd0-a377-4465-bbc8-0f828e36ff4a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:23.693627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 472/921] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Aug 26 15:32:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:23.694858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cd207-5b0e-4116-b1fd-461de43937eb None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:23.695147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cd207-5b0e-4116-b1fd-461de43937eb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:23.695423 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cd207-5b0e-4116-b1fd-461de43937eb None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:23.698519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 473/922] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:23.699301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cd207-5b0e-4116-b1fd-461de43937eb None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:23.710150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 450/923] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:32:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:23.711722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a07a60-e7e8-4aeb-8aab-f1b6e85a8435 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:23.712254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a07a60-e7e8-4aeb-8aab-f1b6e85a8435 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:23.712777 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a07a60-e7e8-4aeb-8aab-f1b6e85a8435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:23.732754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-17a07a60-e7e8-4aeb-8aab-f1b6e85a8435 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:23.734924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-17a07a60-e7e8-4aeb-8aab-f1b6e85a8435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:23.738709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:23.807686 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:23.818219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 451/924] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:23.820133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5a138-09b1-4d22-afa1-f15b67f36a43 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:23.820660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5a138-09b1-4d22-afa1-f15b67f36a43 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:23.821160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5a138-09b1-4d22-afa1-f15b67f36a43 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:23.823058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 452/925] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:23.828969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ba39-ebc2-4b31-9ffc-db890f410579 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:23.829615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ba39-ebc2-4b31-9ffc-db890f410579 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:23.830110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ba39-ebc2-4b31-9ffc-db890f410579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:23.844900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cd207-5b0e-4116-b1fd-461de43937eb None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:23.853015 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-d9c5ba39-ebc2-4b31-9ffc-db890f410579 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:23.855246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d9c5ba39-ebc2-4b31-9ffc-db890f410579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:23.895952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e844a8-720f-40d1-bbde-fa4ff9268090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:23.896609 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e844a8-720f-40d1-bbde-fa4ff9268090 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:23.896885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e844a8-720f-40d1-bbde-fa4ff9268090 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:23.897156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e844a8-720f-40d1-bbde-fa4ff9268090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:23.912572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e844a8-720f-40d1-bbde-fa4ff9268090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:23.914297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e844a8-720f-40d1-bbde-fa4ff9268090 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-e0e844a8-720f-40d1-bbde-fa4ff9268090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:23.933132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:23.934120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:23.935026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:23.935026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:23.935026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:23.937794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 474/926] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:23.943578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e844a8-720f-40d1-bbde-fa4ff9268090 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:23.944817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 453/927] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:24.046945 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e844a8-720f-40d1-bbde-fa4ff9268090 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:24.076952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:24.082063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e43b5-f56a-4d44-ad13-1427ce3e5a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:24.082740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e43b5-f56a-4d44-ad13-1427ce3e5a6e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:24.083024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e43b5-f56a-4d44-ad13-1427ce3e5a6e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:24.083303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e43b5-f56a-4d44-ad13-1427ce3e5a6e None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:24.083978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e43b5-f56a-4d44-ad13-1427ce3e5a6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:24.084374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e43b5-f56a-4d44-ad13-1427ce3e5a6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:24.085747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e43b5-f56a-4d44-ad13-1427ce3e5a6e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:24.098751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 475/928] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:24.100314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5ba39-ebc2-4b31-9ffc-db890f410579 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:24.104085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e43b5-f56a-4d44-ad13-1427ce3e5a6e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:24.169798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:24.208802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e43b5-f56a-4d44-ad13-1427ce3e5a6e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:24.210204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5ba39-ebc2-4b31-9ffc-db890f410579 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:24.248791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6f3d0-fa8a-4642-8dc1-78839dc500d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:24.249423 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6f3d0-fa8a-4642-8dc1-78839dc500d5 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:24.249766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6f3d0-fa8a-4642-8dc1-78839dc500d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:24.250552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6f3d0-fa8a-4642-8dc1-78839dc500d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:24.251947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5980b6-16e9-4643-b9c8-7573096f4b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:24.252593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5980b6-16e9-4643-b9c8-7573096f4b01 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:24.252880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5980b6-16e9-4643-b9c8-7573096f4b01 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:24.253157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5980b6-16e9-4643-b9c8-7573096f4b01 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:24.253752 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5980b6-16e9-4643-b9c8-7573096f4b01 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:24.255110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5980b6-16e9-4643-b9c8-7573096f4b01 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:24.258037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6f3d0-fa8a-4642-8dc1-78839dc500d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:24.259470 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6f3d0-fa8a-4642-8dc1-78839dc500d5 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-7d5980b6-16e9-4643-b9c8-7573096f4b01 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:24.264105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:24.265111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:24.268936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 454/929] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Aug 26 15:32:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:24.269750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5980b6-16e9-4643-b9c8-7573096f4b01 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-e3d6f3d0-fa8a-4642-8dc1-78839dc500d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:24.275369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:24.276651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 476/930] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Aug 26 15:32:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:24.282611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6f3d0-fa8a-4642-8dc1-78839dc500d5 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:24.336279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:24.376175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5980b6-16e9-4643-b9c8-7573096f4b01 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:24.389124 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6f3d0-fa8a-4642-8dc1-78839dc500d5 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:24.405854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:24.411340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32ae02-4861-4b46-b111-7c22a53f1d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:24.416486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32ae02-4861-4b46-b111-7c22a53f1d77 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:24.416486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32ae02-4861-4b46-b111-7c22a53f1d77 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:24.416486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32ae02-4861-4b46-b111-7c22a53f1d77 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:24.416486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32ae02-4861-4b46-b111-7c22a53f1d77 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:24.416486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32ae02-4861-4b46-b111-7c22a53f1d77 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-6b32ae02-4861-4b46-b111-7c22a53f1d77 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:24.424009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:24.425080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:24.426134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 455/931] 10.222.0.98 () {56 vars in 1132 bytes} [Fri Aug 26 15:32:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:24.426969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32ae02-4861-4b46-b111-7c22a53f1d77 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:24.430422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35d490-f786-4cb2-9c4d-eb2da0837f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:24.431045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35d490-f786-4cb2-9c4d-eb2da0837f42 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:24.432201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35d490-f786-4cb2-9c4d-eb2da0837f42 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:24.432201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35d490-f786-4cb2-9c4d-eb2da0837f42 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:24.432398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35d490-f786-4cb2-9c4d-eb2da0837f42 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:24.432572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35d490-f786-4cb2-9c4d-eb2da0837f42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:24.434034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35d490-f786-4cb2-9c4d-eb2da0837f42 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:24.443909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 477/932] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Aug 26 15:32:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:24.443909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35d490-f786-4cb2-9c4d-eb2da0837f42 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:24.542289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32ae02-4861-4b46-b111-7c22a53f1d77 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:24.550389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35d490-f786-4cb2-9c4d-eb2da0837f42 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:24.567633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:24.579273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93558a55-b1e3-478d-a124-58b48923c9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:24.580006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93558a55-b1e3-478d-a124-58b48923c9be None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:24.580296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93558a55-b1e3-478d-a124-58b48923c9be None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:24.582550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93558a55-b1e3-478d-a124-58b48923c9be None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:24.582550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93558a55-b1e3-478d-a124-58b48923c9be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:24.582550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93558a55-b1e3-478d-a124-58b48923c9be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:24.583133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93558a55-b1e3-478d-a124-58b48923c9be None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:24.589649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10de49e-e8e1-437b-bdfe-6eb56e0f9f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:24.590204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10de49e-e8e1-437b-bdfe-6eb56e0f9f69 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:24.590502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10de49e-e8e1-437b-bdfe-6eb56e0f9f69 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:24.590760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10de49e-e8e1-437b-bdfe-6eb56e0f9f69 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:24.591391 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10de49e-e8e1-437b-bdfe-6eb56e0f9f69 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:24.591698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10de49e-e8e1-437b-bdfe-6eb56e0f9f69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:24.593170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10de49e-e8e1-437b-bdfe-6eb56e0f9f69 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:24.602742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 478/933] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Aug 26 15:32:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:24.608434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10de49e-e8e1-437b-bdfe-6eb56e0f9f69 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:24.608533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 456/934] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Aug 26 15:32:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:24.608533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93558a55-b1e3-478d-a124-58b48923c9be None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:24.671247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:24.723245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10de49e-e8e1-437b-bdfe-6eb56e0f9f69 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:24.723768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93558a55-b1e3-478d-a124-58b48923c9be None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:24.766025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5948-292d-41d2-a58d-36fb2867e25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:24.766025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5948-292d-41d2-a58d-36fb2867e25e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:24.766025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5948-292d-41d2-a58d-36fb2867e25e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:24.766442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedc9fa-7c14-402b-9d84-42a34a96e785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:24.766547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5948-292d-41d2-a58d-36fb2867e25e None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:24.766639 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedc9fa-7c14-402b-9d84-42a34a96e785 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:24.766735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a5948-292d-41d2-a58d-36fb2867e25e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:24.766826 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedc9fa-7c14-402b-9d84-42a34a96e785 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:24.766916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a5948-292d-41d2-a58d-36fb2867e25e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:24.767002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedc9fa-7c14-402b-9d84-42a34a96e785 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:24.767002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedc9fa-7c14-402b-9d84-42a34a96e785 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:24.767002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedc9fa-7c14-402b-9d84-42a34a96e785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:24.767204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a5948-292d-41d2-a58d-36fb2867e25e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:24.767274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedc9fa-7c14-402b-9d84-42a34a96e785 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:24.776304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 479/935] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Aug 26 15:32:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:24.776304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedc9fa-7c14-402b-9d84-42a34a96e785 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:24.776541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 457/936] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Aug 26 15:32:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:24.777383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5948-292d-41d2-a58d-36fb2867e25e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:24.827042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:24.883266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5948-292d-41d2-a58d-36fb2867e25e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:24.889908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedc9fa-7c14-402b-9d84-42a34a96e785 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:24.899461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:24.918856 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff2419-6119-4069-9144-0e3d417edf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:24.922851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff2419-6119-4069-9144-0e3d417edf66 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:24.922851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff2419-6119-4069-9144-0e3d417edf66 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:24.922851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff2419-6119-4069-9144-0e3d417edf66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:24.922851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff2419-6119-4069-9144-0e3d417edf66 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:24.923301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff2419-6119-4069-9144-0e3d417edf66 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:24.929364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdf1e5-36a6-44df-a7a2-876de2cc9042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:24.930004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdf1e5-36a6-44df-a7a2-876de2cc9042 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:24.930342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdf1e5-36a6-44df-a7a2-876de2cc9042 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:24.930644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdf1e5-36a6-44df-a7a2-876de2cc9042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:24.941505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdf1e5-36a6-44df-a7a2-876de2cc9042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:24.942957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdf1e5-36a6-44df-a7a2-876de2cc9042 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-37cdf1e5-36a6-44df-a7a2-876de2cc9042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:24.956868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:24.963236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:24.964232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:24.964232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:24.964232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:24.964232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 480/937] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:24.964232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdf1e5-36a6-44df-a7a2-876de2cc9042 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:25.024419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 458/938] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:32:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:25.037016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff2419-6119-4069-9144-0e3d417edf66 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:25.072926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:25.083838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdf1e5-36a6-44df-a7a2-876de2cc9042 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:25.144257 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c69f1a-b47b-4821-8dff-919e91d7ad31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:25.144797 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c69f1a-b47b-4821-8dff-919e91d7ad31 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:25.145084 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c69f1a-b47b-4821-8dff-919e91d7ad31 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:25.145366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c69f1a-b47b-4821-8dff-919e91d7ad31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:25.146391 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c69f1a-b47b-4821-8dff-919e91d7ad31 None admin] RBAC: Adding query filter params (interface=public, service_id=c275bc4cf4774c6fa0889e3b344f2c8e, region_id=RegionOne) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:25.146613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c69f1a-b47b-4821-8dff-919e91d7ad31 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:25.148180 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c69f1a-b47b-4821-8dff-919e91d7ad31 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:25.155969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 481/939] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Aug 26 15:32:24 2022] GET /identity/v3/endpoints?service_id=c275bc4cf4774c6fa0889e3b344f2c8e&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:25.160094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c69f1a-b47b-4821-8dff-919e91d7ad31 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:25.186563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff2419-6119-4069-9144-0e3d417edf66 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:25.237091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b69ea9-72ae-4e07-bc25-e59d8f28cf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:25.237705 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b69ea9-72ae-4e07-bc25-e59d8f28cf10 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:25.237990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b69ea9-72ae-4e07-bc25-e59d8f28cf10 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:25.238266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b69ea9-72ae-4e07-bc25-e59d8f28cf10 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:25.238918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b69ea9-72ae-4e07-bc25-e59d8f28cf10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:25.239338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b69ea9-72ae-4e07-bc25-e59d8f28cf10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:25.240729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b69ea9-72ae-4e07-bc25-e59d8f28cf10 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:25.253880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 459/940] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:25.260452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b69ea9-72ae-4e07-bc25-e59d8f28cf10 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:25.291704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c69f1a-b47b-4821-8dff-919e91d7ad31 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:25.296001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:25.332209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bac53-87f7-4ee8-9019-a7c0792e6e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:25.333191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bac53-87f7-4ee8-9019-a7c0792e6e76 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:25.333696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bac53-87f7-4ee8-9019-a7c0792e6e76 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:25.334046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bac53-87f7-4ee8-9019-a7c0792e6e76 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:25.334767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493bac53-87f7-4ee8-9019-a7c0792e6e76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:25.336424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493bac53-87f7-4ee8-9019-a7c0792e6e76 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:25.343146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 482/941] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Aug 26 15:32:25 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:32:25.381099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b69ea9-72ae-4e07-bc25-e59d8f28cf10 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:25.422536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a94643-99db-4454-baae-3fad3474e576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:25.423503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a94643-99db-4454-baae-3fad3474e576 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:25.423999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a94643-99db-4454-baae-3fad3474e576 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:25.424527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a94643-99db-4454-baae-3fad3474e576 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:25.434082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a94643-99db-4454-baae-3fad3474e576 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:25.437524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a94643-99db-4454-baae-3fad3474e576 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-c3a94643-99db-4454-baae-3fad3474e576 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:25.452110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:25.453469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:25.454409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:25.454409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:25.454409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:25.454409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:25.454409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:25.454409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:25.454409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:25.454409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:25.454409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 460/942] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:25.459680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bac53-87f7-4ee8-9019-a7c0792e6e76 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:25.492718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:25.590819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bac53-87f7-4ee8-9019-a7c0792e6e76 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:25.631683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2da3af-f274-4390-b2fe-37cf8e4f8f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:25.632712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2da3af-f274-4390-b2fe-37cf8e4f8f9d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:25.633200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2da3af-f274-4390-b2fe-37cf8e4f8f9d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:25.633679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2da3af-f274-4390-b2fe-37cf8e4f8f9d None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:25.634591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2da3af-f274-4390-b2fe-37cf8e4f8f9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:25.635128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2da3af-f274-4390-b2fe-37cf8e4f8f9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:25.636773 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2da3af-f274-4390-b2fe-37cf8e4f8f9d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:25.649221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 483/943] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:25.656286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a94643-99db-4454-baae-3fad3474e576 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:25.721995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff45ce-1302-4dca-891d-3739ff0a800d None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:25.722291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff45ce-1302-4dca-891d-3739ff0a800d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:25.722570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff45ce-1302-4dca-891d-3739ff0a800d None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:25.724190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 484/944] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:25.729192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec8c3-d02b-4b1a-9c1b-63a22fb4fea5 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:25.729531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec8c3-d02b-4b1a-9c1b-63a22fb4fea5 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:25.729816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec8c3-d02b-4b1a-9c1b-63a22fb4fea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:25.752012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-214ec8c3-d02b-4b1a-9c1b-63a22fb4fea5 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:25.753349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-214ec8c3-d02b-4b1a-9c1b-63a22fb4fea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:25.754384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:25.764208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a94643-99db-4454-baae-3fad3474e576 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:25.808047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade02fa-9289-4539-a7c1-b14454aef5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:25.809038 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade02fa-9289-4539-a7c1-b14454aef5f9 None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:25.809628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade02fa-9289-4539-a7c1-b14454aef5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:25.810055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade02fa-9289-4539-a7c1-b14454aef5f9 None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/b24a6d0c1dbe49a18b3b8b6732a938be/roles/3fec161072094f87aad16699524908bc` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:25.812565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade02fa-9289-4539-a7c1-b14454aef5f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=b24a6d0c1dbe49a18b3b8b6732a938be, role_id=3fec161072094f87aad16699524908bc)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:25.814273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade02fa-9289-4539-a7c1-b14454aef5f9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:25.828690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 485/945] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:25.870205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 461/946] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:32:25 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/b24a6d0c1dbe49a18b3b8b6732a938be/roles/3fec161072094f87aad16699524908bc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:32:26.098763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec8c3-d02b-4b1a-9c1b-63a22fb4fea5 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:26.110885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:26.203406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec8c3-d02b-4b1a-9c1b-63a22fb4fea5 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:26.239859 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22951d-5cc4-4690-9df2-79d421d8add4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:26.240496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22951d-5cc4-4690-9df2-79d421d8add4 None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:26.240785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22951d-5cc4-4690-9df2-79d421d8add4 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:26.241064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22951d-5cc4-4690-9df2-79d421d8add4 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:26.241715 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22951d-5cc4-4690-9df2-79d421d8add4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:26.243169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22951d-5cc4-4690-9df2-79d421d8add4 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:26.245806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: WARNING keystone.server.flask.common [None req-3f22951d-5cc4-4690-9df2-79d421d8add4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 15:32:26.301958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.sql.core [None req-3f22951d-5cc4-4690-9df2-79d421d8add4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 15:32:26.301958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 15:32:26.301958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [parameters: {'id': '39ed2709420f4bff847d148c4afd356f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 15:32:26.301958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82214) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-3f22951d-5cc4-4690-9df2-79d421d8add4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 39ed2709420f4bff847d148c4afd356f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 39ed2709420f4bff847d148c4afd356f. Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:32:26.310878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application result.read() Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 26 15:32:26.311835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application yield resource Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise self.value Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application session.commit() Aug 26 15:32:26.312884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self.session.flush() Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application compat.raise_( Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 15:32:26.313825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 15:32:26.314767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application result.read() Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:32:26.315589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [parameters: {'id': '39ed2709420f4bff847d148c4afd356f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:26.318301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:26.320468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:26.324716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:26.324716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 26 15:32:26.324716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 26 15:32:26.324716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:26.324716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:26.324716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 26 15:32:26.324716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 26 15:32:26.324716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 39ed2709420f4bff847d148c4afd356f. Aug 26 15:32:26.324716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:26.324716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 486/947] 10.222.0.98 () {60 vars in 1140 bytes} [Fri Aug 26 15:32:26 2022] POST /identity/v3/projects => generated 262 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:26.325292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade02fa-9289-4539-a7c1-b14454aef5f9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:26.427438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade02fa-9289-4539-a7c1-b14454aef5f9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:26.462129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fa79d-a73e-4f0d-b0c7-50915c59bc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:26.463145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fa79d-a73e-4f0d-b0c7-50915c59bc06 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:26.463604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fa79d-a73e-4f0d-b0c7-50915c59bc06 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:26.464082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fa79d-a73e-4f0d-b0c7-50915c59bc06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:26.471686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fa79d-a73e-4f0d-b0c7-50915c59bc06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:26.473428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fa79d-a73e-4f0d-b0c7-50915c59bc06 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-bb1fa79d-a73e-4f0d-b0c7-50915c59bc06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:26.485127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:26.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:26.492517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:26.492517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:26.492517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:26.492517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:26.492517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:26.492517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:26.492517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:26.492517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:26.492517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 462/948] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:26.493817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22951d-5cc4-4690-9df2-79d421d8add4 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:26.564778 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:26.601416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22951d-5cc4-4690-9df2-79d421d8add4 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:26.635426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfd057-6b03-42ff-8206-26fba1ff9b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:26.636075 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd057-6b03-42ff-8206-26fba1ff9b9d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:26.636364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd057-6b03-42ff-8206-26fba1ff9b9d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:26.636645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd057-6b03-42ff-8206-26fba1ff9b9d None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:26.637315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfd057-6b03-42ff-8206-26fba1ff9b9d None admin] RBAC: Adding query filter params (name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:26.637624 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfd057-6b03-42ff-8206-26fba1ff9b9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:26.639272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfd057-6b03-42ff-8206-26fba1ff9b9d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:26.660684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 487/949] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:32:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:27.191114 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c149df9-da3f-4a58-886c-9244a737ff27 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:27.191630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c149df9-da3f-4a58-886c-9244a737ff27 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:27.191743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c149df9-da3f-4a58-886c-9244a737ff27 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:27.193270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 463/950] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:27.198594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa314d8-8d28-4ccf-8323-e4bd700e7e0d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:27.199038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa314d8-8d28-4ccf-8323-e4bd700e7e0d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:27.199233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa314d8-8d28-4ccf-8323-e4bd700e7e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:27.219343 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0aa314d8-8d28-4ccf-8323-e4bd700e7e0d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:27.220892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0aa314d8-8d28-4ccf-8323-e4bd700e7e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:27.294399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 488/951] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:27.356997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0debe3-9236-4849-820d-4633bec445fd None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:27.357277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0debe3-9236-4849-820d-4633bec445fd None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:27.357562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0debe3-9236-4849-820d-4633bec445fd None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:27.359716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 464/952] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:27.365038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e098b9-7404-4d1c-a1c7-d6b5ed1c7fed None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:27.365334 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e098b9-7404-4d1c-a1c7-d6b5ed1c7fed None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:27.365617 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e098b9-7404-4d1c-a1c7-d6b5ed1c7fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:27.384220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f5e098b9-7404-4d1c-a1c7-d6b5ed1c7fed None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:27.385416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f5e098b9-7404-4d1c-a1c7-d6b5ed1c7fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:27.437964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:27.460096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 489/953] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:27.569277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0debe3-9236-4849-820d-4633bec445fd None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:27.585345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:27.680111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0debe3-9236-4849-820d-4633bec445fd None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:27.715308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17381982-27ec-41ba-8cde-423d0cd5037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:27.715940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17381982-27ec-41ba-8cde-423d0cd5037e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:27.716232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17381982-27ec-41ba-8cde-423d0cd5037e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:27.716511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17381982-27ec-41ba-8cde-423d0cd5037e None admin] PATH_INFO: `/v3/services/compute` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:27.717102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17381982-27ec-41ba-8cde-423d0cd5037e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:27.719136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17381982-27ec-41ba-8cde-423d0cd5037e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-17381982-27ec-41ba-8cde-423d0cd5037e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:27.728402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:27.729498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:27.730613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 465/954] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:27.735819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e098b9-7404-4d1c-a1c7-d6b5ed1c7fed None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:27.740603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17381982-27ec-41ba-8cde-423d0cd5037e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:27.830204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:27.850665 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e098b9-7404-4d1c-a1c7-d6b5ed1c7fed None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:27.854846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17381982-27ec-41ba-8cde-423d0cd5037e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:27.889051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5dec6-0f17-47da-b6e2-87fa3726e9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:27.890730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dec6-0f17-47da-b6e2-87fa3726e9e0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:27.891312 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dec6-0f17-47da-b6e2-87fa3726e9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:27.891799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dec6-0f17-47da-b6e2-87fa3726e9e0 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:27.892328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955f095-c52c-4211-b70a-3426867e323c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:27.892932 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5dec6-0f17-47da-b6e2-87fa3726e9e0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:27.893042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f095-c52c-4211-b70a-3426867e323c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:27.893335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f095-c52c-4211-b70a-3426867e323c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:27.893646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f095-c52c-4211-b70a-3426867e323c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:27.893979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5dec6-0f17-47da-b6e2-87fa3726e9e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:27.894234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955f095-c52c-4211-b70a-3426867e323c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:27.895617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955f095-c52c-4211-b70a-3426867e323c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:27.895995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5dec6-0f17-47da-b6e2-87fa3726e9e0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:27.901833 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 490/955] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:32:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-5955f095-c52c-4211-b70a-3426867e323c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:27.905625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:27.906761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:27.907637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:27.908428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5dec6-0f17-47da-b6e2-87fa3726e9e0 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:27.909847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 466/956] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Aug 26 15:32:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:27.916535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955f095-c52c-4211-b70a-3426867e323c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:27.963777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:28.018544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5dec6-0f17-47da-b6e2-87fa3726e9e0 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:28.025514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955f095-c52c-4211-b70a-3426867e323c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:28.053472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841540f6-d8a1-4f0b-9998-dc32f3bc2253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:28.054085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841540f6-d8a1-4f0b-9998-dc32f3bc2253 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:28.054370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841540f6-d8a1-4f0b-9998-dc32f3bc2253 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:28.054647 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841540f6-d8a1-4f0b-9998-dc32f3bc2253 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:28.055280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841540f6-d8a1-4f0b-9998-dc32f3bc2253 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:28.055587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841540f6-d8a1-4f0b-9998-dc32f3bc2253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:28.056994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841540f6-d8a1-4f0b-9998-dc32f3bc2253 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:28.060104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6d0f0-33af-4767-a882-42a79961f77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:28.060700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6d0f0-33af-4767-a882-42a79961f77a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:28.060981 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6d0f0-33af-4767-a882-42a79961f77a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:28.061254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6d0f0-33af-4767-a882-42a79961f77a None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:28.063057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 491/957] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:32:27 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:28.063057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0f386-6c67-4719-b385-59954ab93640 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:28.063350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0f386-6c67-4719-b385-59954ab93640 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:28.063596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0f386-6c67-4719-b385-59954ab93640 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:28.064527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6d0f0-33af-4767-a882-42a79961f77a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:28.065079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6d0f0-33af-4767-a882-42a79961f77a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:28.069523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 492/958] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:28.070336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6d0f0-33af-4767-a882-42a79961f77a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:28.073213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09336d9a-1ce5-4f64-8439-4f4deb39cb0f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:28.073501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09336d9a-1ce5-4f64-8439-4f4deb39cb0f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:28.073777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09336d9a-1ce5-4f64-8439-4f4deb39cb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:28.083766 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 467/959] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Aug 26 15:32:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:28.091943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6d0f0-33af-4767-a882-42a79961f77a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:28.097673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:28.098734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-09336d9a-1ce5-4f64-8439-4f4deb39cb0f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:28.100187 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-09336d9a-1ce5-4f64-8439-4f4deb39cb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:28.176673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 493/960] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:28.228316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6d0f0-33af-4767-a882-42a79961f77a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:28.265437 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e73762-b82a-4c82-92e0-57bbd7e5c5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:28.266050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e73762-b82a-4c82-92e0-57bbd7e5c5a3 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:28.266329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e73762-b82a-4c82-92e0-57bbd7e5c5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:28.266604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e73762-b82a-4c82-92e0-57bbd7e5c5a3 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:28.267235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e73762-b82a-4c82-92e0-57bbd7e5c5a3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:28.267542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e73762-b82a-4c82-92e0-57bbd7e5c5a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:28.269009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e73762-b82a-4c82-92e0-57bbd7e5c5a3 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:28.274422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 468/961] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Aug 26 15:32:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:28.280123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09336d9a-1ce5-4f64-8439-4f4deb39cb0f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:28.344256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:28.409268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09336d9a-1ce5-4f64-8439-4f4deb39cb0f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:28.448605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc02469-a5fe-4988-bbc1-580d3adbc52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:28.449637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc02469-a5fe-4988-bbc1-580d3adbc52e None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:28.450155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc02469-a5fe-4988-bbc1-580d3adbc52e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:28.450687 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc02469-a5fe-4988-bbc1-580d3adbc52e None admin] PATH_INFO: `/v3/endpoints` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:28.451556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc02469-a5fe-4988-bbc1-580d3adbc52e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:28.453907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc02469-a5fe-4988-bbc1-580d3adbc52e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:28.455450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e73762-b82a-4c82-92e0-57bbd7e5c5a3 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:28.548622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 494/962] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:32:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:28.566680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:28.606627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e73762-b82a-4c82-92e0-57bbd7e5c5a3 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:28.656316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772816b0-13f2-4187-848f-7d3105b26448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:28.656963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772816b0-13f2-4187-848f-7d3105b26448 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:28.657247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772816b0-13f2-4187-848f-7d3105b26448 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:28.657525 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772816b0-13f2-4187-848f-7d3105b26448 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:28.670398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772816b0-13f2-4187-848f-7d3105b26448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-772816b0-13f2-4187-848f-7d3105b26448 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:28.676867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:28.677887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:28.678743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 469/963] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Aug 26 15:32:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:28.684279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc02469-a5fe-4988-bbc1-580d3adbc52e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:28.689898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2680e10-9b49-4aba-a9ba-324b3625850e None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:28.690204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2680e10-9b49-4aba-a9ba-324b3625850e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:28.690461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2680e10-9b49-4aba-a9ba-324b3625850e None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:28.692048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 470/964] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:28.703213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee26f16-9726-45fc-a530-5f40ed44ebfb None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:28.703549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee26f16-9726-45fc-a530-5f40ed44ebfb None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:28.703768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee26f16-9726-45fc-a530-5f40ed44ebfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:28.726652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-9ee26f16-9726-45fc-a530-5f40ed44ebfb None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:28.728465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9ee26f16-9726-45fc-a530-5f40ed44ebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:28.811132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:28.813608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 471/965] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:28.816896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc02469-a5fe-4988-bbc1-580d3adbc52e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:28.853400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e51dc-696a-42f9-8d71-d3ea8842665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:28.855827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e51dc-696a-42f9-8d71-d3ea8842665e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:28.855827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e51dc-696a-42f9-8d71-d3ea8842665e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:28.855827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e51dc-696a-42f9-8d71-d3ea8842665e None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:28.855827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e51dc-696a-42f9-8d71-d3ea8842665e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:28.856311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e51dc-696a-42f9-8d71-d3ea8842665e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:28.858011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e51dc-696a-42f9-8d71-d3ea8842665e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:28.867002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 495/966] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Aug 26 15:32:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:28.872441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee26f16-9726-45fc-a530-5f40ed44ebfb None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:28.931367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:28.977673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee26f16-9726-45fc-a530-5f40ed44ebfb None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:29.015435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd47d85-1b1a-4f7d-b904-bf8a868982a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:29.016091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd47d85-1b1a-4f7d-b904-bf8a868982a7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:29.016372 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd47d85-1b1a-4f7d-b904-bf8a868982a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:29.016647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd47d85-1b1a-4f7d-b904-bf8a868982a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:29.027182 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd47d85-1b1a-4f7d-b904-bf8a868982a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:29.028657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd47d85-1b1a-4f7d-b904-bf8a868982a7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-0cd47d85-1b1a-4f7d-b904-bf8a868982a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:29.043350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:29.044388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:29.046001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:29.046001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:29.046001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:29.046001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 472/967] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:29.050645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e51dc-696a-42f9-8d71-d3ea8842665e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:29.117397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd47d85-1b1a-4f7d-b904-bf8a868982a7 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:29.163968 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e51dc-696a-42f9-8d71-d3ea8842665e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:29.209662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b900ec-153d-479d-8591-5ff3962b5d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:29.210692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b900ec-153d-479d-8591-5ff3962b5d3d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:29.211209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b900ec-153d-479d-8591-5ff3962b5d3d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:29.211736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b900ec-153d-479d-8591-5ff3962b5d3d None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:29.212678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b900ec-153d-479d-8591-5ff3962b5d3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:29.213205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b900ec-153d-479d-8591-5ff3962b5d3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:29.214818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b900ec-153d-479d-8591-5ff3962b5d3d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:29.228014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 496/968] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:29.234289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b900ec-153d-479d-8591-5ff3962b5d3d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:29.238666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd47d85-1b1a-4f7d-b904-bf8a868982a7 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:29.251022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:29.263655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:29.283751 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5f604-ab2a-4db2-97dd-473185a96ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:29.284727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5f604-ab2a-4db2-97dd-473185a96ddb None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:29.285182 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5f604-ab2a-4db2-97dd-473185a96ddb None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:29.285626 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5f604-ab2a-4db2-97dd-473185a96ddb None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:29.286610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5f604-ab2a-4db2-97dd-473185a96ddb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:29.288293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5f604-ab2a-4db2-97dd-473185a96ddb None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:29.291107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: WARNING keystone.server.flask.common [None req-5ca5f604-ab2a-4db2-97dd-473185a96ddb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 15:32:29.331145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.sql.core [None req-5ca5f604-ab2a-4db2-97dd-473185a96ddb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 15:32:29.331145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 15:32:29.331145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [parameters: {'user_id': '9af44e92bb6b4fb8b7fc676223105e32', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 15:32:29.331145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82215) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-5ca5f604-ab2a-4db2-97dd-473185a96ddb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application result.read() Aug 26 15:32:29.349000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:32:29.350041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application yield resource Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise self.value Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application session.commit() Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 15:32:29.355805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self.session.flush() Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application compat.raise_( Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 15:32:29.357048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:32:29.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application result.read() Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:32:29.359287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [parameters: {'user_id': '9af44e92bb6b4fb8b7fc676223105e32', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:29.360373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 26 15:32:29.361448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 26 15:32:29.362569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 26 15:32:29.362569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 26 15:32:29.362569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 15:32:29.362569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 15:32:29.362569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 15:32:29.362569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:29.362569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 473/969] 10.222.0.98 () {60 vars in 1131 bytes} [Fri Aug 26 15:32:29 2022] POST /identity/v3/users => generated 160 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:29.364145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5f604-ab2a-4db2-97dd-473185a96ddb None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:29.369125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b900ec-153d-479d-8591-5ff3962b5d3d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:29.411381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc743e38-5b70-4877-b71a-b6015cfe32fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:29.412278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc743e38-5b70-4877-b71a-b6015cfe32fa None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:29.412732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc743e38-5b70-4877-b71a-b6015cfe32fa None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:29.413207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc743e38-5b70-4877-b71a-b6015cfe32fa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:29.421456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc743e38-5b70-4877-b71a-b6015cfe32fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:29.423548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc743e38-5b70-4877-b71a-b6015cfe32fa None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-cc743e38-5b70-4877-b71a-b6015cfe32fa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:29.438016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:29.439218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:29.440429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:29.440429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:29.440429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:29.440429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:29.440429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:29.440429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:29.440429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:29.440429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:29.440429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:32:29.440429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:29.442705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 497/970] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Aug 26 15:32:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:29.447967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc743e38-5b70-4877-b71a-b6015cfe32fa None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:29.503165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5f604-ab2a-4db2-97dd-473185a96ddb None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:29.540696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:29.550729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c929e-6574-4c84-b70c-48cd22d37e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:29.551745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c929e-6574-4c84-b70c-48cd22d37e3e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:29.552472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c929e-6574-4c84-b70c-48cd22d37e3e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:29.552933 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c929e-6574-4c84-b70c-48cd22d37e3e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:29.562957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc743e38-5b70-4877-b71a-b6015cfe32fa None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:29.563747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c929e-6574-4c84-b70c-48cd22d37e3e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:29.566574 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c929e-6574-4c84-b70c-48cd22d37e3e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-d29c929e-6574-4c84-b70c-48cd22d37e3e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:29.588980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:29.590007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:29.590994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:29.590994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:29.590994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:29.590994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:29.590994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:29.590994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:29.590994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:29.590994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:29.590994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:32:29.590994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:29.596284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 474/971] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:32:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:29.604503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c929e-6574-4c84-b70c-48cd22d37e3e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:29.615199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece760d-4efa-4a24-825d-408e4bf53222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:29.615984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece760d-4efa-4a24-825d-408e4bf53222 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:29.616371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece760d-4efa-4a24-825d-408e4bf53222 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:29.616754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece760d-4efa-4a24-825d-408e4bf53222 None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:29.617597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece760d-4efa-4a24-825d-408e4bf53222 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:29.618421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece760d-4efa-4a24-825d-408e4bf53222 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:29.619658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece760d-4efa-4a24-825d-408e4bf53222 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:29.643267 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 498/972] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Aug 26 15:32:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:29.644427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece760d-4efa-4a24-825d-408e4bf53222 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:29.649816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:29.753325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c929e-6574-4c84-b70c-48cd22d37e3e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:29.761339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece760d-4efa-4a24-825d-408e4bf53222 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:29.795759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cbf49-bafd-4522-b608-e1c760a7c861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:29.796635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cbf49-bafd-4522-b608-e1c760a7c861 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:29.797319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cbf49-bafd-4522-b608-e1c760a7c861 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:29.797899 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cbf49-bafd-4522-b608-e1c760a7c861 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:29.798748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cbf49-bafd-4522-b608-e1c760a7c861 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:29.799510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cbf49-bafd-4522-b608-e1c760a7c861 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:29.801387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cbf49-bafd-4522-b608-e1c760a7c861 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:29.813484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293579d-8e79-45ec-b34e-2625c69c654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:29.814513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293579d-8e79-45ec-b34e-2625c69c654c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:29.814822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293579d-8e79-45ec-b34e-2625c69c654c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:29.815218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293579d-8e79-45ec-b34e-2625c69c654c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:29.830592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:29.852056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 475/973] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Aug 26 15:32:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:29.909313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293579d-8e79-45ec-b34e-2625c69c654c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:29.911208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293579d-8e79-45ec-b34e-2625c69c654c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:30.027956 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 499/974] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:29 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 15:32:30.031184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cbf49-bafd-4522-b608-e1c760a7c861 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:30.127706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:30.149633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cbf49-bafd-4522-b608-e1c760a7c861 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:30.160373 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5a236-0148-4c0f-99ed-6b0d12a738d6 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:30.160667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5a236-0148-4c0f-99ed-6b0d12a738d6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:30.160866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5a236-0148-4c0f-99ed-6b0d12a738d6 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:30.162348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 500/975] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:30.167427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c413a2e-1ff7-4e72-a650-7e06334b555a None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:30.167746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c413a2e-1ff7-4e72-a650-7e06334b555a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:30.168152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c413a2e-1ff7-4e72-a650-7e06334b555a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:30.188233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde58716-c095-4bbb-a47f-4e6732fef88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:30.188553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1c413a2e-1ff7-4e72-a650-7e06334b555a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:30.188866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde58716-c095-4bbb-a47f-4e6732fef88c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:30.189146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde58716-c095-4bbb-a47f-4e6732fef88c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:30.189426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde58716-c095-4bbb-a47f-4e6732fef88c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:30.189758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1c413a2e-1ff7-4e72-a650-7e06334b555a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:30.200105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde58716-c095-4bbb-a47f-4e6732fef88c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:30.201646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde58716-c095-4bbb-a47f-4e6732fef88c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-fde58716-c095-4bbb-a47f-4e6732fef88c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:30.218205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:30.219404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:30.220521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:30.220521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:30.220521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:30.220521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 476/976] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:30.224771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde58716-c095-4bbb-a47f-4e6732fef88c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:30.232220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:30.265829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 501/977] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:30.329406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde58716-c095-4bbb-a47f-4e6732fef88c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:30.365317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769573b2-cd00-44a4-8d35-7d267412741f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:30.365919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769573b2-cd00-44a4-8d35-7d267412741f None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:30.366197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769573b2-cd00-44a4-8d35-7d267412741f None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:30.366530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769573b2-cd00-44a4-8d35-7d267412741f None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:30.367170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769573b2-cd00-44a4-8d35-7d267412741f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:30.367470 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769573b2-cd00-44a4-8d35-7d267412741f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:30.368848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769573b2-cd00-44a4-8d35-7d267412741f None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:30.376281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:30.386337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 477/978] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:30.389032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c413a2e-1ff7-4e72-a650-7e06334b555a None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:30.487664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c413a2e-1ff7-4e72-a650-7e06334b555a None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:30.521361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875274d-9332-4938-aff8-cd055a03ccce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:30.522156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875274d-9332-4938-aff8-cd055a03ccce None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:30.522533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875274d-9332-4938-aff8-cd055a03ccce None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:30.522904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875274d-9332-4938-aff8-cd055a03ccce None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:30.529771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875274d-9332-4938-aff8-cd055a03ccce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:30.531553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875274d-9332-4938-aff8-cd055a03ccce None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:30.535667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769573b2-cd00-44a4-8d35-7d267412741f None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-d875274d-9332-4938-aff8-cd055a03ccce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:30.544162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:30.545272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:30.547139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:30.547139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:30.547139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:30.547139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:30.547139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:30.547139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:30.547139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:30.547139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:30.547701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 502/979] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:30.552739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875274d-9332-4938-aff8-cd055a03ccce None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:30.584331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:30.639027 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769573b2-cd00-44a4-8d35-7d267412741f None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:30.652475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875274d-9332-4938-aff8-cd055a03ccce None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:30.673819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646f497-769e-4302-90cc-668d7cf82be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:30.674427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646f497-769e-4302-90cc-668d7cf82be9 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:30.674709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646f497-769e-4302-90cc-668d7cf82be9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:30.674985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646f497-769e-4302-90cc-668d7cf82be9 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:30.675638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646f497-769e-4302-90cc-668d7cf82be9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:30.677047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646f497-769e-4302-90cc-668d7cf82be9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:30.686799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95c4b0-51b6-41d6-9e28-bfa3db743efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:30.687390 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95c4b0-51b6-41d6-9e28-bfa3db743efe None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:30.687669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95c4b0-51b6-41d6-9e28-bfa3db743efe None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:30.687966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95c4b0-51b6-41d6-9e28-bfa3db743efe None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:30.688647 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95c4b0-51b6-41d6-9e28-bfa3db743efe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:30.688949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95c4b0-51b6-41d6-9e28-bfa3db743efe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:30.690352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95c4b0-51b6-41d6-9e28-bfa3db743efe None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:30.701404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 503/980] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:30.702749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ebd56-e618-4cb5-a07a-9e7c5edb25e2 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:30.703036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ebd56-e618-4cb5-a07a-9e7c5edb25e2 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:30.703338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ebd56-e618-4cb5-a07a-9e7c5edb25e2 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:30.704539 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 504/981] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:30.707629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ebd56-e618-4cb5-a07a-9e7c5edb25e2 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:30.749083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 478/982] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:32:30 2022] POST /identity/v3/services => generated 242 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:32:30.750428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf96a2-3da1-45f4-bd41-bdf76cf315b4 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:30.750771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf96a2-3da1-45f4-bd41-bdf76cf315b4 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:30.751126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf96a2-3da1-45f4-bd41-bdf76cf315b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:30.774656 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-bbaf96a2-3da1-45f4-bd41-bdf76cf315b4 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:30.776309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-bbaf96a2-3da1-45f4-bd41-bdf76cf315b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:30.808051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:30.827037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ebd56-e618-4cb5-a07a-9e7c5edb25e2 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:30.839303 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:30.863894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 479/983] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:30.873781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef15364-a080-456f-b4bc-741e99fb69d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:30.874403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef15364-a080-456f-b4bc-741e99fb69d6 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:30.874686 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef15364-a080-456f-b4bc-741e99fb69d6 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:30.874959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef15364-a080-456f-b4bc-741e99fb69d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:30.875703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef15364-a080-456f-b4bc-741e99fb69d6 None admin] RBAC: Adding query filter params (role.id=3fec161072094f87aad16699524908bc, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=b24a6d0c1dbe49a18b3b8b6732a938be) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:30.876044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef15364-a080-456f-b4bc-741e99fb69d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:30.877521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef15364-a080-456f-b4bc-741e99fb69d6 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:30.896370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 505/984] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:30 2022] GET /identity/v3/role_assignments?user.id=b24a6d0c1dbe49a18b3b8b6732a938be&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=3fec161072094f87aad16699524908bc => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:31.126235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf96a2-3da1-45f4-bd41-bdf76cf315b4 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:31.228694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf96a2-3da1-45f4-bd41-bdf76cf315b4 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:31.265476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce715f8-d664-4528-94dd-50b98a4d2cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:31.266101 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce715f8-d664-4528-94dd-50b98a4d2cc3 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:31.266380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce715f8-d664-4528-94dd-50b98a4d2cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:31.266653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce715f8-d664-4528-94dd-50b98a4d2cc3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:31.267290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce715f8-d664-4528-94dd-50b98a4d2cc3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:31.268670 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce715f8-d664-4528-94dd-50b98a4d2cc3 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-2ce715f8-d664-4528-94dd-50b98a4d2cc3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:31.277276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:31.278282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:31.281262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 480/985] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Aug 26 15:32:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:31.285361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef15364-a080-456f-b4bc-741e99fb69d6 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:31.389685 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef15364-a080-456f-b4bc-741e99fb69d6 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:31.425361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafd97b-9e4a-4884-90b9-5d85ec80ae13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:31.426260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafd97b-9e4a-4884-90b9-5d85ec80ae13 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:31.426745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafd97b-9e4a-4884-90b9-5d85ec80ae13 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:31.427206 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafd97b-9e4a-4884-90b9-5d85ec80ae13 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:31.428043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafd97b-9e4a-4884-90b9-5d85ec80ae13 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:31.428589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafd97b-9e4a-4884-90b9-5d85ec80ae13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:31.430114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafd97b-9e4a-4884-90b9-5d85ec80ae13 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:31.435010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 506/986] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Aug 26 15:32:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:31.441290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce715f8-d664-4528-94dd-50b98a4d2cc3 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:31.495549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:31.551242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce715f8-d664-4528-94dd-50b98a4d2cc3 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:31.596460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfd381-1179-46fc-805a-6082c5dbe64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:31.597078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfd381-1179-46fc-805a-6082c5dbe64c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:31.597366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfd381-1179-46fc-805a-6082c5dbe64c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:31.597647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfd381-1179-46fc-805a-6082c5dbe64c None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:31.600053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfd381-1179-46fc-805a-6082c5dbe64c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:31.600053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfd381-1179-46fc-805a-6082c5dbe64c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:31.600053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfd381-1179-46fc-805a-6082c5dbe64c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:31.610789 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 481/987] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Aug 26 15:32:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:31.611710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafd97b-9e4a-4884-90b9-5d85ec80ae13 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:31.628547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:31.713002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafd97b-9e4a-4884-90b9-5d85ec80ae13 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:31.748054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c8760-e66f-454e-95d1-b33700b6ab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:31.748692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c8760-e66f-454e-95d1-b33700b6ab8e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:31.748976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c8760-e66f-454e-95d1-b33700b6ab8e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:31.749250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c8760-e66f-454e-95d1-b33700b6ab8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:31.749954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c8760-e66f-454e-95d1-b33700b6ab8e None admin] RBAC: Adding query filter params (interface=admin, service_id=c275bc4cf4774c6fa0889e3b344f2c8e, region_id=RegionOne) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:31.750264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c8760-e66f-454e-95d1-b33700b6ab8e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:31.751635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c8760-e66f-454e-95d1-b33700b6ab8e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:31.761333 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 507/988] 10.222.0.98 () {56 vars in 1249 bytes} [Fri Aug 26 15:32:31 2022] GET /identity/v3/endpoints?service_id=c275bc4cf4774c6fa0889e3b344f2c8e&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:31.762421 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfd381-1179-46fc-805a-6082c5dbe64c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:31.885594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfd381-1179-46fc-805a-6082c5dbe64c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:31.908092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:31.931428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a9944-cd0e-4aca-b5e8-2296fa0ef952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:31.932074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a9944-cd0e-4aca-b5e8-2296fa0ef952 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:31.932363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a9944-cd0e-4aca-b5e8-2296fa0ef952 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:31.932641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a9944-cd0e-4aca-b5e8-2296fa0ef952 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:31.933277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a9944-cd0e-4aca-b5e8-2296fa0ef952 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:31.934647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a9944-cd0e-4aca-b5e8-2296fa0ef952 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:31.947959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 482/989] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Aug 26 15:32:31 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:32:32.695294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f4a0d-090d-4459-b228-2203f3b8b58e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:32.695845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f4a0d-090d-4459-b228-2203f3b8b58e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:32.695845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f4a0d-090d-4459-b228-2203f3b8b58e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:32.697370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 508/990] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:32.702433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255435e5-db0e-47fa-ad73-b045c21193a2 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:32.703094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255435e5-db0e-47fa-ad73-b045c21193a2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:32.703094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255435e5-db0e-47fa-ad73-b045c21193a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:32.720640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-255435e5-db0e-47fa-ad73-b045c21193a2 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:32.721825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-255435e5-db0e-47fa-ad73-b045c21193a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:32.790224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 483/991] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:32.840870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda8296-ddb4-445b-b1a5-d744f66ecfaa None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:32.841169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda8296-ddb4-445b-b1a5-d744f66ecfaa None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:32.841424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda8296-ddb4-445b-b1a5-d744f66ecfaa None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:32.842919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 509/992] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:32.847831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c31dfa-bd5c-422d-a2ed-8e818f6e1edf None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:32.848157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c31dfa-bd5c-422d-a2ed-8e818f6e1edf None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:32.848436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c31dfa-bd5c-422d-a2ed-8e818f6e1edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:32.866685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-81c31dfa-bd5c-422d-a2ed-8e818f6e1edf None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:32.867788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-81c31dfa-bd5c-422d-a2ed-8e818f6e1edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:32.937023 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 484/993] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:33.063680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda8296-ddb4-445b-b1a5-d744f66ecfaa None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:33.144904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:33.167116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda8296-ddb4-445b-b1a5-d744f66ecfaa None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:33.201615 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20115e3d-b157-4236-bf4c-2e0b0765017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:33.202237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115e3d-b157-4236-bf4c-2e0b0765017b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:33.202527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115e3d-b157-4236-bf4c-2e0b0765017b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:33.202802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115e3d-b157-4236-bf4c-2e0b0765017b None admin] PATH_INFO: `/v3/services/compute` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:33.203392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20115e3d-b157-4236-bf4c-2e0b0765017b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:33.204179 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c31dfa-bd5c-422d-a2ed-8e818f6e1edf None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:33.204910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20115e3d-b157-4236-bf4c-2e0b0765017b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-20115e3d-b157-4236-bf4c-2e0b0765017b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:33.218832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:33.219923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:33.220810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 510/994] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:33.225626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20115e3d-b157-4236-bf4c-2e0b0765017b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:33.250892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:33.314956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c31dfa-bd5c-422d-a2ed-8e818f6e1edf None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:33.335835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20115e3d-b157-4236-bf4c-2e0b0765017b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:33.352596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3e804-2d72-4329-8c27-c89bf149ad35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:33.353239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3e804-2d72-4329-8c27-c89bf149ad35 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:33.353524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3e804-2d72-4329-8c27-c89bf149ad35 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:33.354006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3e804-2d72-4329-8c27-c89bf149ad35 None admin] PATH_INFO: `/v3/services/image` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:33.354376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3e804-2d72-4329-8c27-c89bf149ad35 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:33.355742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3e804-2d72-4329-8c27-c89bf149ad35 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-a1d3e804-2d72-4329-8c27-c89bf149ad35 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:33.365005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:33.366118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:33.367034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 485/995] 10.222.0.98 () {56 vars in 1105 bytes} [Fri Aug 26 15:32:33 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:33.371938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3e804-2d72-4329-8c27-c89bf149ad35 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:33.374090 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951def5-fa9b-4d71-8b2b-56bd8235af8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:33.374702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951def5-fa9b-4d71-8b2b-56bd8235af8e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:33.375013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951def5-fa9b-4d71-8b2b-56bd8235af8e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:33.375293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951def5-fa9b-4d71-8b2b-56bd8235af8e None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:33.375963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951def5-fa9b-4d71-8b2b-56bd8235af8e None admin] RBAC: Adding query filter params (name=compute) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:33.376270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951def5-fa9b-4d71-8b2b-56bd8235af8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:33.377668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951def5-fa9b-4d71-8b2b-56bd8235af8e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:33.382800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 511/996] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:32:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:33.388205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951def5-fa9b-4d71-8b2b-56bd8235af8e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:33.398749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:33.486650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3e804-2d72-4329-8c27-c89bf149ad35 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:33.490073 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:33.497616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951def5-fa9b-4d71-8b2b-56bd8235af8e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:33.525647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7980f53-0514-4964-9766-9bd7b76fa1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:33.526399 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7980f53-0514-4964-9766-9bd7b76fa1ab None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:33.526764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7980f53-0514-4964-9766-9bd7b76fa1ab None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:33.527117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7980f53-0514-4964-9766-9bd7b76fa1ab None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:33.527819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7980f53-0514-4964-9766-9bd7b76fa1ab None admin] RBAC: Adding query filter params (name=image) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:33.528159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7980f53-0514-4964-9766-9bd7b76fa1ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:33.530112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7980f53-0514-4964-9766-9bd7b76fa1ab None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:33.535527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 486/997] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:33.540919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7980f53-0514-4964-9766-9bd7b76fa1ab None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:33.541397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b7fcc-1bbc-40c6-9001-e7eb380d8333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:33.542122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b7fcc-1bbc-40c6-9001-e7eb380d8333 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:33.542407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b7fcc-1bbc-40c6-9001-e7eb380d8333 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:33.542683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b7fcc-1bbc-40c6-9001-e7eb380d8333 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:33.543314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b7fcc-1bbc-40c6-9001-e7eb380d8333 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:33.543620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b7fcc-1bbc-40c6-9001-e7eb380d8333 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:33.545037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b7fcc-1bbc-40c6-9001-e7eb380d8333 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:33.555182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 512/998] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:32:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:33.556259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b7fcc-1bbc-40c6-9001-e7eb380d8333 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:33.642177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:33.669364 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7980f53-0514-4964-9766-9bd7b76fa1ab None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:33.673841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b7fcc-1bbc-40c6-9001-e7eb380d8333 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:33.712713 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe73f8d-5d8b-40d7-bcfa-c2cbb71b207a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:33.713254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48485239-5ac5-4b3d-9c5c-4afa6ac26d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:33.713633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485239-5ac5-4b3d-9c5c-4afa6ac26d6f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:33.713714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73f8d-5d8b-40d7-bcfa-c2cbb71b207a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:33.713714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73f8d-5d8b-40d7-bcfa-c2cbb71b207a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:33.713844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485239-5ac5-4b3d-9c5c-4afa6ac26d6f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:33.713913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73f8d-5d8b-40d7-bcfa-c2cbb71b207a None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:33.713998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485239-5ac5-4b3d-9c5c-4afa6ac26d6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:33.714457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe73f8d-5d8b-40d7-bcfa-c2cbb71b207a None admin] RBAC: Adding query filter params (type=image) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:33.714555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48485239-5ac5-4b3d-9c5c-4afa6ac26d6f None admin] RBAC: Adding query filter params (interface=public, service_id=68682c4227f04189b91e6a33408f6ccf, region_id=RegionOne) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:33.714756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe73f8d-5d8b-40d7-bcfa-c2cbb71b207a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:33.714861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48485239-5ac5-4b3d-9c5c-4afa6ac26d6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:33.716212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48485239-5ac5-4b3d-9c5c-4afa6ac26d6f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:33.716309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe73f8d-5d8b-40d7-bcfa-c2cbb71b207a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:33.726446 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 487/999] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:33 2022] GET /identity/v3/services?type=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:33.726635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 513/1000] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Aug 26 15:32:33 2022] GET /identity/v3/endpoints?service_id=68682c4227f04189b91e6a33408f6ccf&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:33.727651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe73f8d-5d8b-40d7-bcfa-c2cbb71b207a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:33.762583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:33.839824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe73f8d-5d8b-40d7-bcfa-c2cbb71b207a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:33.844607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef25f81-f2c7-4049-9adc-d3b3a839ff91 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:33.844911 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef25f81-f2c7-4049-9adc-d3b3a839ff91 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:33.845171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef25f81-f2c7-4049-9adc-d3b3a839ff91 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:33.847902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 514/1001] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:33.852119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3b08d-348f-4ece-9264-cfc3161fad74 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:33.852412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3b08d-348f-4ece-9264-cfc3161fad74 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:33.852694 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3b08d-348f-4ece-9264-cfc3161fad74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:33.873723 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-8ac3b08d-348f-4ece-9264-cfc3161fad74 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:33.874935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8ac3b08d-348f-4ece-9264-cfc3161fad74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:33.880510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a30362-0b68-4381-b227-f926acdc2e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:33.881123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a30362-0b68-4381-b227-f926acdc2e3b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:33.881465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a30362-0b68-4381-b227-f926acdc2e3b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:33.881680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a30362-0b68-4381-b227-f926acdc2e3b None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:33.882286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a30362-0b68-4381-b227-f926acdc2e3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:33.883651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a30362-0b68-4381-b227-f926acdc2e3b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:33.894208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 488/1002] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Aug 26 15:32:33 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:32:33.957202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 515/1003] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:34.244770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a30362-0b68-4381-b227-f926acdc2e3b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:34.344074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:34.352561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a30362-0b68-4381-b227-f926acdc2e3b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:34.386878 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d6ec2-4169-401d-8edb-5d7c2e02aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:34.388436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d6ec2-4169-401d-8edb-5d7c2e02aeaa None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:34.389487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d6ec2-4169-401d-8edb-5d7c2e02aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:34.389939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d6ec2-4169-401d-8edb-5d7c2e02aeaa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:34.390743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d6ec2-4169-401d-8edb-5d7c2e02aeaa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:34.392317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d6ec2-4169-401d-8edb-5d7c2e02aeaa None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-bc3d6ec2-4169-401d-8edb-5d7c2e02aeaa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:34.401434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:34.402621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:32:34.403463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:34.404250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 489/1004] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Aug 26 15:32:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:34.409336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3b08d-348f-4ece-9264-cfc3161fad74 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:34.473560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:34.516205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3b08d-348f-4ece-9264-cfc3161fad74 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:34.549261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb43c1e-d28b-440e-aa99-d96fdea28cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:34.549872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb43c1e-d28b-440e-aa99-d96fdea28cd0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:34.550156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb43c1e-d28b-440e-aa99-d96fdea28cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:34.550431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb43c1e-d28b-440e-aa99-d96fdea28cd0 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:34.551063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb43c1e-d28b-440e-aa99-d96fdea28cd0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:34.551365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb43c1e-d28b-440e-aa99-d96fdea28cd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:34.552756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb43c1e-d28b-440e-aa99-d96fdea28cd0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:34.557696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 516/1005] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Aug 26 15:32:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:34.563728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d6ec2-4169-401d-8edb-5d7c2e02aeaa None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:34.673593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d6ec2-4169-401d-8edb-5d7c2e02aeaa None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:34.710594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e43db-fa63-4f19-be2a-a4fd9d3f0abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:34.711201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e43db-fa63-4f19-be2a-a4fd9d3f0abe None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:34.711487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e43db-fa63-4f19-be2a-a4fd9d3f0abe None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:34.711756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e43db-fa63-4f19-be2a-a4fd9d3f0abe None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:34.712414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e43db-fa63-4f19-be2a-a4fd9d3f0abe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:34.712715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e43db-fa63-4f19-be2a-a4fd9d3f0abe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:34.714079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e43db-fa63-4f19-be2a-a4fd9d3f0abe None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:34.718854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 490/1006] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Aug 26 15:32:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:34.724502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb43c1e-d28b-440e-aa99-d96fdea28cd0 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:34.829752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb43c1e-d28b-440e-aa99-d96fdea28cd0 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:34.855632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:34.864432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78eaee-4b5c-42ed-a008-6a60c1e8c353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:34.867291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78eaee-4b5c-42ed-a008-6a60c1e8c353 None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:34.867291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78eaee-4b5c-42ed-a008-6a60c1e8c353 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:34.867291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78eaee-4b5c-42ed-a008-6a60c1e8c353 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:34.867291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78eaee-4b5c-42ed-a008-6a60c1e8c353 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:34.867696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78eaee-4b5c-42ed-a008-6a60c1e8c353 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:34.937977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 517/1007] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:32:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:35.662071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4187fac-855e-485c-a717-46cc61c643fd None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:35.662390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4187fac-855e-485c-a717-46cc61c643fd None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:35.662633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4187fac-855e-485c-a717-46cc61c643fd None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:35.664172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 491/1008] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:35.669148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87032f71-c268-48ef-8a57-96c612ade92a None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:35.669446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87032f71-c268-48ef-8a57-96c612ade92a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:35.669731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87032f71-c268-48ef-8a57-96c612ade92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:35.687976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-87032f71-c268-48ef-8a57-96c612ade92a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:35.689170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-87032f71-c268-48ef-8a57-96c612ade92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:35.762628 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 518/1009] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:35.827889 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0340a-e74b-4b3a-8ab9-c6ac1fe34a61 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:35.828384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0340a-e74b-4b3a-8ab9-c6ac1fe34a61 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:35.828489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0340a-e74b-4b3a-8ab9-c6ac1fe34a61 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:35.829940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 492/1010] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:35.834888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c4d1c-bb0b-4cdb-b5bc-178db3281e95 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:35.835186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c4d1c-bb0b-4cdb-b5bc-178db3281e95 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:35.835469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c4d1c-bb0b-4cdb-b5bc-178db3281e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:35.854174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-dc4c4d1c-bb0b-4cdb-b5bc-178db3281e95 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:35.855382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-dc4c4d1c-bb0b-4cdb-b5bc-178db3281e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:35.924283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 519/1011] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:36.034556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0340a-e74b-4b3a-8ab9-c6ac1fe34a61 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:36.063091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:36.139242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0340a-e74b-4b3a-8ab9-c6ac1fe34a61 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:36.174037 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83d86c-abea-49e2-99b4-6d907ceb4006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:36.174635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83d86c-abea-49e2-99b4-6d907ceb4006 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:36.174915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83d86c-abea-49e2-99b4-6d907ceb4006 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:36.175188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83d86c-abea-49e2-99b4-6d907ceb4006 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:36.175821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83d86c-abea-49e2-99b4-6d907ceb4006 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:36.177212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83d86c-abea-49e2-99b4-6d907ceb4006 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:36.194343 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c4d1c-bb0b-4cdb-b5bc-178db3281e95 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:36.239678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:36.280924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 493/1012] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:32:36 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:36.295755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c4d1c-bb0b-4cdb-b5bc-178db3281e95 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:36.346908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80c1dd-d2ba-4a25-8c1a-840fa3a9a969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:36.347558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80c1dd-d2ba-4a25-8c1a-840fa3a9a969 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:36.347841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80c1dd-d2ba-4a25-8c1a-840fa3a9a969 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:36.348146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80c1dd-d2ba-4a25-8c1a-840fa3a9a969 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:36.348742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80c1dd-d2ba-4a25-8c1a-840fa3a9a969 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:36.350126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80c1dd-d2ba-4a25-8c1a-840fa3a9a969 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-ee80c1dd-d2ba-4a25-8c1a-840fa3a9a969 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:36.360134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:36.361234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:32:36.362103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:36.365269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 520/1013] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:36.367474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83d86c-abea-49e2-99b4-6d907ceb4006 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:36.461497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:36.479757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83d86c-abea-49e2-99b4-6d907ceb4006 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:36.514469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a61bbf-8c2b-445e-8346-5a022e63e54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:36.515805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a61bbf-8c2b-445e-8346-5a022e63e54e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:36.516365 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a61bbf-8c2b-445e-8346-5a022e63e54e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:36.516647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a61bbf-8c2b-445e-8346-5a022e63e54e None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:36.517282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a61bbf-8c2b-445e-8346-5a022e63e54e None admin] RBAC: Adding query filter params (name=compute) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:36.517591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a61bbf-8c2b-445e-8346-5a022e63e54e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:36.519025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a61bbf-8c2b-445e-8346-5a022e63e54e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:36.528289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 494/1014] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:32:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:36.530031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80c1dd-d2ba-4a25-8c1a-840fa3a9a969 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:36.643385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80c1dd-d2ba-4a25-8c1a-840fa3a9a969 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:36.646562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:36.688262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53ef5-36f2-46e9-9d9f-39a44f8c6ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:36.689291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53ef5-36f2-46e9-9d9f-39a44f8c6ef5 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:36.689782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53ef5-36f2-46e9-9d9f-39a44f8c6ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:36.690278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53ef5-36f2-46e9-9d9f-39a44f8c6ef5 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:36.691405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53ef5-36f2-46e9-9d9f-39a44f8c6ef5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:36.691956 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53ef5-36f2-46e9-9d9f-39a44f8c6ef5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:36.694436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53ef5-36f2-46e9-9d9f-39a44f8c6ef5 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:36.702024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 521/1015] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Aug 26 15:32:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:36.707796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a61bbf-8c2b-445e-8346-5a022e63e54e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:36.809352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a61bbf-8c2b-445e-8346-5a022e63e54e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:36.844100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28180694-67a5-4a93-8ba9-e68a663a0f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:36.844722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28180694-67a5-4a93-8ba9-e68a663a0f4c None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:36.844988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28180694-67a5-4a93-8ba9-e68a663a0f4c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:36.845267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28180694-67a5-4a93-8ba9-e68a663a0f4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:36.845936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28180694-67a5-4a93-8ba9-e68a663a0f4c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:36.847299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28180694-67a5-4a93-8ba9-e68a663a0f4c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:36.862661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a724c0e-2fad-481c-bf84-5db52e1889fe None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:36.863172 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a724c0e-2fad-481c-bf84-5db52e1889fe None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:36.863372 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a724c0e-2fad-481c-bf84-5db52e1889fe None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:36.864831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 522/1016] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:36.869573 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac717eeb-47c7-4808-8bc9-a9c3d397d41e None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:36.869854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac717eeb-47c7-4808-8bc9-a9c3d397d41e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:36.870635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac717eeb-47c7-4808-8bc9-a9c3d397d41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:36.888004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ac717eeb-47c7-4808-8bc9-a9c3d397d41e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:36.889934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ac717eeb-47c7-4808-8bc9-a9c3d397d41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:36.924958 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 495/1017] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:32:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:36.970492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 523/1018] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:37.236625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28180694-67a5-4a93-8ba9-e68a663a0f4c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:37.240623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:37.338900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28180694-67a5-4a93-8ba9-e68a663a0f4c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:37.372198 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e6516-e30f-4100-9653-abccf1c9b9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:37.373386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e6516-e30f-4100-9653-abccf1c9b9df None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:37.373668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e6516-e30f-4100-9653-abccf1c9b9df None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:37.373934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e6516-e30f-4100-9653-abccf1c9b9df None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:37.375021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e6516-e30f-4100-9653-abccf1c9b9df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:37.376474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e6516-e30f-4100-9653-abccf1c9b9df None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:37.378229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 496/1019] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:37.383882 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac717eeb-47c7-4808-8bc9-a9c3d397d41e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:37.480771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:37.488679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac717eeb-47c7-4808-8bc9-a9c3d397d41e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:37.522388 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04244002-5afd-44c8-945f-2b7ed401da0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:37.522995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04244002-5afd-44c8-945f-2b7ed401da0a None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:37.523273 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04244002-5afd-44c8-945f-2b7ed401da0a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:37.523545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04244002-5afd-44c8-945f-2b7ed401da0a None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:37.524340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04244002-5afd-44c8-945f-2b7ed401da0a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:37.525759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04244002-5afd-44c8-945f-2b7ed401da0a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:37.612516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 524/1020] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:32:37 2022] POST /identity/v3/projects => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:38.238925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c2257-3228-4a2d-b55a-9df1d30f40a5 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:38.239470 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c2257-3228-4a2d-b55a-9df1d30f40a5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:38.239470 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c2257-3228-4a2d-b55a-9df1d30f40a5 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:38.240905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 497/1021] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:38.246151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c573b3-be0d-4174-a39f-38e9e8b87036 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:38.246562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c573b3-be0d-4174-a39f-38e9e8b87036 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:38.246738 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c573b3-be0d-4174-a39f-38e9e8b87036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:38.264817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e4c573b3-be0d-4174-a39f-38e9e8b87036 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:38.265997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e4c573b3-be0d-4174-a39f-38e9e8b87036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:38.333178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 525/1022] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:38.601855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c2257-3228-4a2d-b55a-9df1d30f40a5 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:38.677289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:38.709687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c2257-3228-4a2d-b55a-9df1d30f40a5 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:38.745461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f7707-9bd5-4946-b9f4-7962c1a86ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:38.746092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f7707-9bd5-4946-b9f4-7962c1a86ba3 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:38.746374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f7707-9bd5-4946-b9f4-7962c1a86ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:38.746648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f7707-9bd5-4946-b9f4-7962c1a86ba3 None admin] PATH_INFO: `/v3/services/image` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:38.747225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f7707-9bd5-4946-b9f4-7962c1a86ba3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:38.749177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f7707-9bd5-4946-b9f4-7962c1a86ba3 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-637f7707-9bd5-4946-b9f4-7962c1a86ba3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:38.757712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:38.758738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:38.759932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 498/1023] 10.222.0.98 () {56 vars in 1105 bytes} [Fri Aug 26 15:32:38 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:38.764678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c573b3-be0d-4174-a39f-38e9e8b87036 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:38.839559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:38.862100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c573b3-be0d-4174-a39f-38e9e8b87036 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:38.896441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa48e9-c3ca-4fa4-a2d1-fda8ba7fcc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:38.897678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa48e9-c3ca-4fa4-a2d1-fda8ba7fcc59 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:38.897968 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa48e9-c3ca-4fa4-a2d1-fda8ba7fcc59 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:38.898247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa48e9-c3ca-4fa4-a2d1-fda8ba7fcc59 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:38.898878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa48e9-c3ca-4fa4-a2d1-fda8ba7fcc59 None admin] RBAC: Adding query filter params (name=image) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:38.899189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa48e9-c3ca-4fa4-a2d1-fda8ba7fcc59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:38.900595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa48e9-c3ca-4fa4-a2d1-fda8ba7fcc59 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:38.906332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 526/1024] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:38.912045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f7707-9bd5-4946-b9f4-7962c1a86ba3 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:38.930152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda9cc5-249a-4bde-be68-5ef3ebadacb6 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:38.930467 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda9cc5-249a-4bde-be68-5ef3ebadacb6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:38.930866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda9cc5-249a-4bde-be68-5ef3ebadacb6 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:38.932467 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 527/1025] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:38.938030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf48a-4dba-4817-bc5c-ebecab5e55cc None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:38.938319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf48a-4dba-4817-bc5c-ebecab5e55cc None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:38.938597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf48a-4dba-4817-bc5c-ebecab5e55cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:38.956293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-365bf48a-4dba-4817-bc5c-ebecab5e55cc None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:38.957493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-365bf48a-4dba-4817-bc5c-ebecab5e55cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:39.023058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f7707-9bd5-4946-b9f4-7962c1a86ba3 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:39.030655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 528/1026] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:39.059460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8b3c3-5b11-4a39-a56d-32084a259cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:39.060115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8b3c3-5b11-4a39-a56d-32084a259cf4 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:39.060396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8b3c3-5b11-4a39-a56d-32084a259cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:39.060664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8b3c3-5b11-4a39-a56d-32084a259cf4 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:39.061282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8b3c3-5b11-4a39-a56d-32084a259cf4 None admin] RBAC: Adding query filter params (type=image) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:39.061577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8b3c3-5b11-4a39-a56d-32084a259cf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:39.062937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8b3c3-5b11-4a39-a56d-32084a259cf4 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:39.065380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:39.073202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 499/1027] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:39.074862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bf48a-4dba-4817-bc5c-ebecab5e55cc None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:39.147373 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:39.178622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bf48a-4dba-4817-bc5c-ebecab5e55cc None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:39.212719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ec6ca-06ab-4825-b896-2a7638c80b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:39.213330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec6ca-06ab-4825-b896-2a7638c80b65 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:39.213616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec6ca-06ab-4825-b896-2a7638c80b65 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:39.213892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec6ca-06ab-4825-b896-2a7638c80b65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:39.214588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ec6ca-06ab-4825-b896-2a7638c80b65 None admin] RBAC: Adding query filter params (interface=public, service_id=4551ddf20741462686129ae834503042, region_id=RegionOne) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:39.214900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ec6ca-06ab-4825-b896-2a7638c80b65 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:39.216837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ec6ca-06ab-4825-b896-2a7638c80b65 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:39.226456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 529/1028] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Aug 26 15:32:39 2022] GET /identity/v3/endpoints?service_id=4551ddf20741462686129ae834503042&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:39.227794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8b3c3-5b11-4a39-a56d-32084a259cf4 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:39.299923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ec6ca-06ab-4825-b896-2a7638c80b65 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:39.338294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8b3c3-5b11-4a39-a56d-32084a259cf4 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:39.379980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464e72e-b18d-4aee-bf4d-aee8083c400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:39.380858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464e72e-b18d-4aee-bf4d-aee8083c400b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:39.381138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464e72e-b18d-4aee-bf4d-aee8083c400b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:39.381407 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464e72e-b18d-4aee-bf4d-aee8083c400b None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:39.382158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464e72e-b18d-4aee-bf4d-aee8083c400b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:39.384185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464e72e-b18d-4aee-bf4d-aee8083c400b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:39.393099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 500/1029] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Aug 26 15:32:39 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:32:39.406162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ec6ca-06ab-4825-b896-2a7638c80b65 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:39.446023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b4716-66f7-439b-85bb-c720f90e18bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:39.446943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b4716-66f7-439b-85bb-c720f90e18bd None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:39.447413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b4716-66f7-439b-85bb-c720f90e18bd None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:39.447940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b4716-66f7-439b-85bb-c720f90e18bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:39.451062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:39.462209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b4716-66f7-439b-85bb-c720f90e18bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-5d0b4716-66f7-439b-85bb-c720f90e18bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:39.467962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:39.469190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:39.470253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:39.472785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 530/1030] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Aug 26 15:32:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:39.478637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464e72e-b18d-4aee-bf4d-aee8083c400b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:39.549179 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:39.561204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1095d91-fb8b-4442-9b6e-c6e9ac8ab7cb None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:39.561493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1095d91-fb8b-4442-9b6e-c6e9ac8ab7cb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:39.561765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1095d91-fb8b-4442-9b6e-c6e9ac8ab7cb None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:39.563108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 531/1031] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:39.568522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb3d6-9fd2-450b-b656-c4d50608829b None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:39.568819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb3d6-9fd2-450b-b656-c4d50608829b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:39.569104 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb3d6-9fd2-450b-b656-c4d50608829b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:39.589291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-eb3eb3d6-9fd2-450b-b656-c4d50608829b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:39.590495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-eb3eb3d6-9fd2-450b-b656-c4d50608829b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:39.591024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464e72e-b18d-4aee-bf4d-aee8083c400b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:39.630222 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d7cdb-e8d3-4d5b-afd7-df995150a11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:39.631629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d7cdb-e8d3-4d5b-afd7-df995150a11b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:39.632117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d7cdb-e8d3-4d5b-afd7-df995150a11b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:39.632228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d7cdb-e8d3-4d5b-afd7-df995150a11b None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:39.632874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d7cdb-e8d3-4d5b-afd7-df995150a11b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:39.633177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d7cdb-e8d3-4d5b-afd7-df995150a11b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:39.634541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d7cdb-e8d3-4d5b-afd7-df995150a11b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:39.647635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 501/1032] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Aug 26 15:32:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:39.648573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d7cdb-e8d3-4d5b-afd7-df995150a11b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:39.667035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 532/1033] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:39.753127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d7cdb-e8d3-4d5b-afd7-df995150a11b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:39.779335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:39.787848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406f144-6474-4e76-8eeb-d7327a6e5cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:39.795985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406f144-6474-4e76-8eeb-d7327a6e5cc5 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:39.795985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406f144-6474-4e76-8eeb-d7327a6e5cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:39.795985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406f144-6474-4e76-8eeb-d7327a6e5cc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:39.800192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406f144-6474-4e76-8eeb-d7327a6e5cc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:39.801945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406f144-6474-4e76-8eeb-d7327a6e5cc5 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-b406f144-6474-4e76-8eeb-d7327a6e5cc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:39.815973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:39.816943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:39.817875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:39.817875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:39.817875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:39.820206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 502/1034] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:39.823084 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3eb3d6-9fd2-450b-b656-c4d50608829b None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:39.926567 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3eb3d6-9fd2-450b-b656-c4d50608829b None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:39.940317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406f144-6474-4e76-8eeb-d7327a6e5cc5 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:39.952210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:39.963831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3e5ed-cead-41d9-9358-6be92515c74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:39.964683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3e5ed-cead-41d9-9358-6be92515c74f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:39.966719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3e5ed-cead-41d9-9358-6be92515c74f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:39.966719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3e5ed-cead-41d9-9358-6be92515c74f None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:39.966719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3e5ed-cead-41d9-9358-6be92515c74f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:39.967234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3e5ed-cead-41d9-9358-6be92515c74f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:39.968626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3e5ed-cead-41d9-9358-6be92515c74f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:39.984206 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 533/1035] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:39.986533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3e5ed-cead-41d9-9358-6be92515c74f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:40.049034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406f144-6474-4e76-8eeb-d7327a6e5cc5 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:40.084238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a6821-e5d6-4fbd-a886-2c639a2d75a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:40.085282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a6821-e5d6-4fbd-a886-2c639a2d75a7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:40.085568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a6821-e5d6-4fbd-a886-2c639a2d75a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:40.085992 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a6821-e5d6-4fbd-a886-2c639a2d75a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:40.087100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a6821-e5d6-4fbd-a886-2c639a2d75a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:40.088577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a6821-e5d6-4fbd-a886-2c639a2d75a7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:40.090140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 503/1036] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:40.096732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a6821-e5d6-4fbd-a886-2c639a2d75a7 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:40.101711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3e5ed-cead-41d9-9358-6be92515c74f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:40.136235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005727f-0cdc-40fc-9782-7f907bc3e5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:40.136858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005727f-0cdc-40fc-9782-7f907bc3e5e6 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:40.137133 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005727f-0cdc-40fc-9782-7f907bc3e5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:40.137411 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005727f-0cdc-40fc-9782-7f907bc3e5e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:40.145016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:40.145495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005727f-0cdc-40fc-9782-7f907bc3e5e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:40.147365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005727f-0cdc-40fc-9782-7f907bc3e5e6 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-9005727f-0cdc-40fc-9782-7f907bc3e5e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:40.158962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:40.162143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:40.163130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:40.163130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:40.163130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:40.163130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:40.163130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:40.163130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:40.163130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:40.163130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:40.163130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:32:40.163130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:40.163130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 534/1037] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Aug 26 15:32:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:40.165693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005727f-0cdc-40fc-9782-7f907bc3e5e6 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:40.202634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a6821-e5d6-4fbd-a886-2c639a2d75a7 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:40.212360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:40.240486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc468e-825c-41c6-80e1-c8197d9d2a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:40.241056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc468e-825c-41c6-80e1-c8197d9d2a38 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:40.241338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc468e-825c-41c6-80e1-c8197d9d2a38 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:40.241617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc468e-825c-41c6-80e1-c8197d9d2a38 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:40.242321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc468e-825c-41c6-80e1-c8197d9d2a38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:40.244547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc468e-825c-41c6-80e1-c8197d9d2a38 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:40.275089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005727f-0cdc-40fc-9782-7f907bc3e5e6 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:40.308965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088055e0-44e2-4547-a992-9118ffc61a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:40.309591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088055e0-44e2-4547-a992-9118ffc61a32 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:40.309864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088055e0-44e2-4547-a992-9118ffc61a32 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:40.310160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088055e0-44e2-4547-a992-9118ffc61a32 None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:40.310899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088055e0-44e2-4547-a992-9118ffc61a32 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:40.311208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088055e0-44e2-4547-a992-9118ffc61a32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:40.312657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088055e0-44e2-4547-a992-9118ffc61a32 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:40.329932 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 535/1038] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Aug 26 15:32:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:40.335616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088055e0-44e2-4547-a992-9118ffc61a32 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:40.364366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 504/1039] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:32:40 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:32:40.446505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088055e0-44e2-4547-a992-9118ffc61a32 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:40.448226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:40.486880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f77f9-b58d-40cc-b310-e34ac610d18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:40.487418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f77f9-b58d-40cc-b310-e34ac610d18d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:40.487708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f77f9-b58d-40cc-b310-e34ac610d18d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:40.488031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f77f9-b58d-40cc-b310-e34ac610d18d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:40.564290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f77f9-b58d-40cc-b310-e34ac610d18d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:40.565750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f77f9-b58d-40cc-b310-e34ac610d18d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:40.673938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 536/1040] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:40 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 15:32:40.679400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc468e-825c-41c6-80e1-c8197d9d2a38 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:40.780851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc468e-825c-41c6-80e1-c8197d9d2a38 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:40.782345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:40.816454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1f820-560c-4c59-b171-23aa701ac3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:40.817228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1f820-560c-4c59-b171-23aa701ac3d6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:40.817609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1f820-560c-4c59-b171-23aa701ac3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:40.817979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1f820-560c-4c59-b171-23aa701ac3d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:40.831447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1f820-560c-4c59-b171-23aa701ac3d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:40.834953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1f820-560c-4c59-b171-23aa701ac3d6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-24d1f820-560c-4c59-b171-23aa701ac3d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:40.851457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:40.852544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:40.860664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:40.860664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:40.860664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:40.860664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 505/1041] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:40.860910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f77f9-b58d-40cc-b310-e34ac610d18d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:40.862362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:40.962029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f77f9-b58d-40cc-b310-e34ac610d18d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:40.995528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a44774-a224-4b37-aab8-eba898ba869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:40.996163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a44774-a224-4b37-aab8-eba898ba869f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:40.996451 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a44774-a224-4b37-aab8-eba898ba869f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:40.996731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a44774-a224-4b37-aab8-eba898ba869f None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:40.997407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a44774-a224-4b37-aab8-eba898ba869f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:40.997716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a44774-a224-4b37-aab8-eba898ba869f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:40.999092 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a44774-a224-4b37-aab8-eba898ba869f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:41.022149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 537/1042] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:41.024344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1f820-560c-4c59-b171-23aa701ac3d6 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:41.129346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1f820-560c-4c59-b171-23aa701ac3d6 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:41.162705 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27fb9f-8070-4b8a-bf10-c842fa996a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:41.163759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27fb9f-8070-4b8a-bf10-c842fa996a5e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:41.164277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27fb9f-8070-4b8a-bf10-c842fa996a5e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:41.164755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27fb9f-8070-4b8a-bf10-c842fa996a5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:41.176339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27fb9f-8070-4b8a-bf10-c842fa996a5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:41.177815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27fb9f-8070-4b8a-bf10-c842fa996a5e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-4d27fb9f-8070-4b8a-bf10-c842fa996a5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:41.189118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:41.190017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:41.190962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:41.190962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:41.190962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:41.190962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:41.190962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:41.190962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:41.190962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:41.190962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:41.195047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 506/1043] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:41.196421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a44774-a224-4b37-aab8-eba898ba869f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:41.268950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:41.308776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a44774-a224-4b37-aab8-eba898ba869f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:41.327003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa805a8-6118-4036-8d45-a0d5ca81d0fb None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:41.327285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa805a8-6118-4036-8d45-a0d5ca81d0fb None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:41.327556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa805a8-6118-4036-8d45-a0d5ca81d0fb None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:41.328926 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 507/1044] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:41.334063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaecb5-271c-4a5a-8244-8c9523f5fe50 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:41.334422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaecb5-271c-4a5a-8244-8c9523f5fe50 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:41.334713 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaecb5-271c-4a5a-8244-8c9523f5fe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:41.346748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c28b3-e9ff-4901-b8b5-37b37914bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:41.347354 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c28b3-e9ff-4901-b8b5-37b37914bcde None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:41.347641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c28b3-e9ff-4901-b8b5-37b37914bcde None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:41.347942 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c28b3-e9ff-4901-b8b5-37b37914bcde None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:41.348637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c28b3-e9ff-4901-b8b5-37b37914bcde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:41.349465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c28b3-e9ff-4901-b8b5-37b37914bcde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:41.351180 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c28b3-e9ff-4901-b8b5-37b37914bcde None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:41.354112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-20eaecb5-271c-4a5a-8244-8c9523f5fe50 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:41.355322 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-20eaecb5-271c-4a5a-8244-8c9523f5fe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:41.365482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:41.366277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 538/1045] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:41.372852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c28b3-e9ff-4901-b8b5-37b37914bcde None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:41.436729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 508/1046] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:41.493254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c28b3-e9ff-4901-b8b5-37b37914bcde None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:41.516707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:41.528948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ae6d8-1cf6-488e-a31c-0498056c9777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:41.529557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ae6d8-1cf6-488e-a31c-0498056c9777 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:41.529835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ae6d8-1cf6-488e-a31c-0498056c9777 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:41.530114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ae6d8-1cf6-488e-a31c-0498056c9777 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:41.530833 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ae6d8-1cf6-488e-a31c-0498056c9777 None admin] RBAC: Adding query filter params (role.id=3fec161072094f87aad16699524908bc, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=d1b01c21c38949938141eefdc4939385) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:41.531146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ae6d8-1cf6-488e-a31c-0498056c9777 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:41.532597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ae6d8-1cf6-488e-a31c-0498056c9777 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:41.543915 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 539/1047] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:41 2022] GET /identity/v3/role_assignments?user.id=d1b01c21c38949938141eefdc4939385&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=3fec161072094f87aad16699524908bc => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:41.707578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaecb5-271c-4a5a-8244-8c9523f5fe50 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:41.812362 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaecb5-271c-4a5a-8244-8c9523f5fe50 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:41.848263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d7d15-90a8-4ab4-b589-5728fd80e354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:41.848902 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d7d15-90a8-4ab4-b589-5728fd80e354 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:41.849185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d7d15-90a8-4ab4-b589-5728fd80e354 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:41.849461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d7d15-90a8-4ab4-b589-5728fd80e354 None admin] PATH_INFO: `/v3/services/image` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:41.850048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d7d15-90a8-4ab4-b589-5728fd80e354 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:41.851411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d7d15-90a8-4ab4-b589-5728fd80e354 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-467d7d15-90a8-4ab4-b589-5728fd80e354 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:41.860127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:41.861134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:32:41.862009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:41.863949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 509/1048] 10.222.0.98 () {56 vars in 1105 bytes} [Fri Aug 26 15:32:41 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:41.867217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ae6d8-1cf6-488e-a31c-0498056c9777 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:41.969057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ae6d8-1cf6-488e-a31c-0498056c9777 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:42.006216 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127a674-61c0-4794-af12-05afd55456cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:42.006832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127a674-61c0-4794-af12-05afd55456cf None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:42.007118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127a674-61c0-4794-af12-05afd55456cf None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:42.007397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127a674-61c0-4794-af12-05afd55456cf None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:42.008064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127a674-61c0-4794-af12-05afd55456cf None admin] RBAC: Adding query filter params (name=image) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:42.008375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127a674-61c0-4794-af12-05afd55456cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:42.009753 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127a674-61c0-4794-af12-05afd55456cf None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:42.015522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 540/1049] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:42.021170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d7d15-90a8-4ab4-b589-5728fd80e354 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:42.034681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:42.126668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d7d15-90a8-4ab4-b589-5728fd80e354 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:42.160553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3c3fa-4fce-4795-877f-c189bab0da02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:42.161166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3c3fa-4fce-4795-877f-c189bab0da02 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:42.161452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3c3fa-4fce-4795-877f-c189bab0da02 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:42.161732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3c3fa-4fce-4795-877f-c189bab0da02 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:42.162363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3c3fa-4fce-4795-877f-c189bab0da02 None admin] RBAC: Adding query filter params (type=image) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:42.162669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3c3fa-4fce-4795-877f-c189bab0da02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:42.164067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3c3fa-4fce-4795-877f-c189bab0da02 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:42.169658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 510/1050] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:42.175440 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127a674-61c0-4794-af12-05afd55456cf None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:42.241555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:42.292888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127a674-61c0-4794-af12-05afd55456cf None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:42.326976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff2e6e-8f0b-4227-bfdb-ea54897c7322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:42.327581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff2e6e-8f0b-4227-bfdb-ea54897c7322 None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:42.327880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff2e6e-8f0b-4227-bfdb-ea54897c7322 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:42.328167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff2e6e-8f0b-4227-bfdb-ea54897c7322 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:42.328837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff2e6e-8f0b-4227-bfdb-ea54897c7322 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:42.330217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff2e6e-8f0b-4227-bfdb-ea54897c7322 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:42.391100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dffeab-0e7a-4d12-bab2-e1c682ac0cfd None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:42.391632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dffeab-0e7a-4d12-bab2-e1c682ac0cfd None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:42.391937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dffeab-0e7a-4d12-bab2-e1c682ac0cfd None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:42.393684 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 511/1051] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:42.400180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214b972-7625-4462-bb4f-cd3c5de52041 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:42.400611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214b972-7625-4462-bb4f-cd3c5de52041 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:42.400956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214b972-7625-4462-bb4f-cd3c5de52041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:42.424321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-d214b972-7625-4462-bb4f-cd3c5de52041 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:42.425527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d214b972-7625-4462-bb4f-cd3c5de52041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:42.433513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 541/1052] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:32:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:42.504283 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:42.512013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 512/1053] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:42.778129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff2e6e-8f0b-4227-bfdb-ea54897c7322 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:42.883513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff2e6e-8f0b-4227-bfdb-ea54897c7322 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:42.891413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:42.920114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea832c3-0558-4286-91de-b2f216932052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:42.921002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea832c3-0558-4286-91de-b2f216932052 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:42.921284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea832c3-0558-4286-91de-b2f216932052 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:42.921560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea832c3-0558-4286-91de-b2f216932052 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:42.931489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea832c3-0558-4286-91de-b2f216932052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-0ea832c3-0558-4286-91de-b2f216932052 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:42.936728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:42.937706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:42.938582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:42.942905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 542/1054] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:32:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:42.944200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214b972-7625-4462-bb4f-cd3c5de52041 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:43.056493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214b972-7625-4462-bb4f-cd3c5de52041 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:43.090376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584e8a9-f4dd-4028-a6cf-1fa3a1d62abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:43.091014 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584e8a9-f4dd-4028-a6cf-1fa3a1d62abc None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:43.091294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584e8a9-f4dd-4028-a6cf-1fa3a1d62abc None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:43.091568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584e8a9-f4dd-4028-a6cf-1fa3a1d62abc None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:43.092252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584e8a9-f4dd-4028-a6cf-1fa3a1d62abc None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:43.092556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584e8a9-f4dd-4028-a6cf-1fa3a1d62abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:43.093939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584e8a9-f4dd-4028-a6cf-1fa3a1d62abc None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:43.107507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 513/1055] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:32:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:43.108861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea832c3-0558-4286-91de-b2f216932052 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:43.214385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea832c3-0558-4286-91de-b2f216932052 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:43.250768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a8f80-6533-4e26-a493-d006deabee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:43.251399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a8f80-6533-4e26-a493-d006deabee6d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:43.251682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a8f80-6533-4e26-a493-d006deabee6d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:43.251983 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a8f80-6533-4e26-a493-d006deabee6d None admin] PATH_INFO: `/v3/users/a59d38223c774a5581332246f664c37d` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:43.265068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a8f80-6533-4e26-a493-d006deabee6d None admin] RBAC: Authorizing `identity:get_user(user_id=a59d38223c774a5581332246f664c37d)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:43.266508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a8f80-6533-4e26-a493-d006deabee6d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:43.268323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 543/1056] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:32:43 2022] GET /identity/v3/users/a59d38223c774a5581332246f664c37d => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:43.273997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584e8a9-f4dd-4028-a6cf-1fa3a1d62abc None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:43.334655 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:43.379313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584e8a9-f4dd-4028-a6cf-1fa3a1d62abc None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:43.414773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ffd54-e65e-4e45-bf7f-59867fd0c522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:43.415558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ffd54-e65e-4e45-bf7f-59867fd0c522 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:43.415728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ffd54-e65e-4e45-bf7f-59867fd0c522 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:43.416087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ffd54-e65e-4e45-bf7f-59867fd0c522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:43.452534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167af671-a351-4f13-a82f-2c4d785d6178 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:43.453045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167af671-a351-4f13-a82f-2c4d785d6178 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:43.453150 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167af671-a351-4f13-a82f-2c4d785d6178 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:43.454660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 544/1057] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:43.459640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363d6a-894f-4b2b-9cc6-abe210f651d3 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:43.467444 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363d6a-894f-4b2b-9cc6-abe210f651d3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:43.467444 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363d6a-894f-4b2b-9cc6-abe210f651d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:43.478867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-49363d6a-894f-4b2b-9cc6-abe210f651d3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:43.480140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-49363d6a-894f-4b2b-9cc6-abe210f651d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:43.492338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ffd54-e65e-4e45-bf7f-59867fd0c522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:43.493792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ffd54-e65e-4e45-bf7f-59867fd0c522 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:43.499442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:43.555243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:43.560088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 545/1058] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:43.609960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 514/1059] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:32:43.615632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49363d6a-894f-4b2b-9cc6-abe210f651d3 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:43.718270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49363d6a-894f-4b2b-9cc6-abe210f651d3 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:43.761866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a51eeb-8ab5-4da1-b6af-98e8961190fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:43.762969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a51eeb-8ab5-4da1-b6af-98e8961190fb None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:43.763249 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a51eeb-8ab5-4da1-b6af-98e8961190fb None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:43.763519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a51eeb-8ab5-4da1-b6af-98e8961190fb None admin] PATH_INFO: `/v3/projects/512b66fecdb843789f5257338378b59a` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:43.764656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a51eeb-8ab5-4da1-b6af-98e8961190fb None admin] RBAC: Authorizing `identity:get_project(project_id=512b66fecdb843789f5257338378b59a)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:43.766120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a51eeb-8ab5-4da1-b6af-98e8961190fb None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:43.768085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 546/1060] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:32:43 2022] GET /identity/v3/projects/512b66fecdb843789f5257338378b59a => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:43.774251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ffd54-e65e-4e45-bf7f-59867fd0c522 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:43.854242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a51eeb-8ab5-4da1-b6af-98e8961190fb None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:43.856940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:43.907029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ffd54-e65e-4e45-bf7f-59867fd0c522 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:43.942737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def3bc8-e862-405d-ba8a-44adf431f68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:43.942980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def3bc8-e862-405d-ba8a-44adf431f68a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:43.943091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def3bc8-e862-405d-ba8a-44adf431f68a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:43.943362 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def3bc8-e862-405d-ba8a-44adf431f68a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:43.944166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def3bc8-e862-405d-ba8a-44adf431f68a None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=512b66fecdb843789f5257338378b59a, user.id=a59d38223c774a5581332246f664c37d) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:43.944482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def3bc8-e862-405d-ba8a-44adf431f68a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:43.945907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def3bc8-e862-405d-ba8a-44adf431f68a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:43.963900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 515/1061] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:43 2022] GET /identity/v3/role_assignments?user.id=a59d38223c774a5581332246f664c37d&scope.project.id=512b66fecdb843789f5257338378b59a&role.id=6ba5a5add2174983be37258aad1ee404 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:43.964108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a51eeb-8ab5-4da1-b6af-98e8961190fb None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:44.007545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ffb02-6b05-4ce5-b0d8-e94b62e726a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:44.008623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ffb02-6b05-4ce5-b0d8-e94b62e726a1 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:44.009089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ffb02-6b05-4ce5-b0d8-e94b62e726a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:44.009541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ffb02-6b05-4ce5-b0d8-e94b62e726a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:44.029932 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ffb02-6b05-4ce5-b0d8-e94b62e726a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-224ffb02-6b05-4ce5-b0d8-e94b62e726a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:44.039101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:44.040202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:44.041069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:44.041713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 547/1062] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Aug 26 15:32:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:44.050402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def3bc8-e862-405d-ba8a-44adf431f68a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:44.084128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:44.167116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def3bc8-e862-405d-ba8a-44adf431f68a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:44.204946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453de04-4287-4259-a8aa-4137f5fa7f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:44.205555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453de04-4287-4259-a8aa-4137f5fa7f1d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:44.205839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453de04-4287-4259-a8aa-4137f5fa7f1d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:44.206115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453de04-4287-4259-a8aa-4137f5fa7f1d None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:44.206858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453de04-4287-4259-a8aa-4137f5fa7f1d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:44.207168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453de04-4287-4259-a8aa-4137f5fa7f1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:44.208568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453de04-4287-4259-a8aa-4137f5fa7f1d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:44.217327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 516/1063] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Aug 26 15:32:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:44.223155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ffb02-6b05-4ce5-b0d8-e94b62e726a1 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:44.312175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:44.330966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ffb02-6b05-4ce5-b0d8-e94b62e726a1 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:44.366871 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c31f5-e250-4027-b8e9-467aa7f8df18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:44.367497 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c31f5-e250-4027-b8e9-467aa7f8df18 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:44.367776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c31f5-e250-4027-b8e9-467aa7f8df18 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:44.368078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c31f5-e250-4027-b8e9-467aa7f8df18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:44.378600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252dbd7-f8bd-40d6-bb2f-186276aca8da None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:44.378885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252dbd7-f8bd-40d6-bb2f-186276aca8da None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:44.379160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252dbd7-f8bd-40d6-bb2f-186276aca8da None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:44.380809 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 517/1064] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:44.381056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c31f5-e250-4027-b8e9-467aa7f8df18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:44.382884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c31f5-e250-4027-b8e9-467aa7f8df18 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:44.385778 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4251b38-b235-4666-9f6f-45510ffc1831 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:44.386064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4251b38-b235-4666-9f6f-45510ffc1831 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:44.386354 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4251b38-b235-4666-9f6f-45510ffc1831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-6c3c31f5-e250-4027-b8e9-467aa7f8df18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:44.403714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:44.405361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:44.406500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:44.406500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:44.406500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:44.406500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 548/1065] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:44.407060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e4251b38-b235-4666-9f6f-45510ffc1831 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:44.408299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e4251b38-b235-4666-9f6f-45510ffc1831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:44.414231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c31f5-e250-4027-b8e9-467aa7f8df18 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:44.486447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 518/1066] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:44.530637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c31f5-e250-4027-b8e9-467aa7f8df18 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:44.565031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf1e9d-98cc-470d-84ce-dbecc5e8652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:44.565633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf1e9d-98cc-470d-84ce-dbecc5e8652c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:44.565918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf1e9d-98cc-470d-84ce-dbecc5e8652c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:44.566191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf1e9d-98cc-470d-84ce-dbecc5e8652c None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:44.566888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf1e9d-98cc-470d-84ce-dbecc5e8652c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:44.567191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf1e9d-98cc-470d-84ce-dbecc5e8652c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:44.568737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf1e9d-98cc-470d-84ce-dbecc5e8652c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:44.570330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:44.586197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 549/1067] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:44.593560 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4251b38-b235-4666-9f6f-45510ffc1831 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:44.618825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:44.704520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4251b38-b235-4666-9f6f-45510ffc1831 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:44.738415 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9485d49-008d-40a9-b632-43736d49ab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:44.738971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9485d49-008d-40a9-b632-43736d49ab91 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:44.739256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9485d49-008d-40a9-b632-43736d49ab91 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:44.739533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9485d49-008d-40a9-b632-43736d49ab91 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:44.746406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9485d49-008d-40a9-b632-43736d49ab91 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:44.747858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9485d49-008d-40a9-b632-43736d49ab91 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-e9485d49-008d-40a9-b632-43736d49ab91 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:44.759191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:44.760253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:44.761358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:44.761358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:44.761358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:44.761358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:44.761358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:44.761358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:44.761358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:44.761358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:44.761358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:32:44.761358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:44.761358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 519/1068] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Aug 26 15:32:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:44.762000 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf1e9d-98cc-470d-84ce-dbecc5e8652c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:44.766947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9485d49-008d-40a9-b632-43736d49ab91 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:44.856820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:44.880534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9485d49-008d-40a9-b632-43736d49ab91 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:44.896637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf1e9d-98cc-470d-84ce-dbecc5e8652c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:44.917837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0e346-62fb-4dc2-9126-c85c18f531df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:44.918439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0e346-62fb-4dc2-9126-c85c18f531df None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:44.918744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0e346-62fb-4dc2-9126-c85c18f531df None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:44.919016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0e346-62fb-4dc2-9126-c85c18f531df None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:44.919753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0e346-62fb-4dc2-9126-c85c18f531df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:44.920431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0e346-62fb-4dc2-9126-c85c18f531df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:44.921779 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0e346-62fb-4dc2-9126-c85c18f531df None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:44.932810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc8606-3a04-46d4-9d51-6b7115a25447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:44.933416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc8606-3a04-46d4-9d51-6b7115a25447 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:44.933700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc8606-3a04-46d4-9d51-6b7115a25447 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:44.933975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc8606-3a04-46d4-9d51-6b7115a25447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:44.945753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 520/1069] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Aug 26 15:32:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:44.946606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0e346-62fb-4dc2-9126-c85c18f531df None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:45.011091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:45.021888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc8606-3a04-46d4-9d51-6b7115a25447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:45.032551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc8606-3a04-46d4-9d51-6b7115a25447 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:45.068278 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0e346-62fb-4dc2-9126-c85c18f531df None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:45.103206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef7f06-b7e2-4226-9e13-e02fdd75b445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:45.103833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef7f06-b7e2-4226-9e13-e02fdd75b445 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:45.104146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef7f06-b7e2-4226-9e13-e02fdd75b445 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:45.104422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef7f06-b7e2-4226-9e13-e02fdd75b445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:45.106891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:45.114980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef7f06-b7e2-4226-9e13-e02fdd75b445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:45.116465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef7f06-b7e2-4226-9e13-e02fdd75b445 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-4aef7f06-b7e2-4226-9e13-e02fdd75b445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:45.130827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:45.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:45.133350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:45.133350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:45.133350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:45.133350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 521/1070] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:45.139492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 550/1071] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:32:45.139627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef7f06-b7e2-4226-9e13-e02fdd75b445 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:45.140999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc8606-3a04-46d4-9d51-6b7115a25447 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:45.247169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef7f06-b7e2-4226-9e13-e02fdd75b445 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:45.255267 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:45.263326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc8606-3a04-46d4-9d51-6b7115a25447 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:45.281440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb07985-2989-4447-b6fc-5b26f9dac5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:45.282114 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07985-2989-4447-b6fc-5b26f9dac5bd None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:45.282335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07985-2989-4447-b6fc-5b26f9dac5bd None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:45.282616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07985-2989-4447-b6fc-5b26f9dac5bd None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:45.283259 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb07985-2989-4447-b6fc-5b26f9dac5bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:45.283563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb07985-2989-4447-b6fc-5b26f9dac5bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:45.284970 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb07985-2989-4447-b6fc-5b26f9dac5bd None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:45.296736 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 522/1072] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:45.302269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb07985-2989-4447-b6fc-5b26f9dac5bd None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:45.307330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956fbe5-6c1a-4565-a06c-9b5c113cf9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:45.307986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956fbe5-6c1a-4565-a06c-9b5c113cf9cf None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:45.308271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956fbe5-6c1a-4565-a06c-9b5c113cf9cf None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:45.308546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956fbe5-6c1a-4565-a06c-9b5c113cf9cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:45.319518 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956fbe5-6c1a-4565-a06c-9b5c113cf9cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:45.321004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956fbe5-6c1a-4565-a06c-9b5c113cf9cf None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-5956fbe5-6c1a-4565-a06c-9b5c113cf9cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:45.339545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:45.341495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:45.342424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:45.342424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:45.342424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:45.342799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 551/1073] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:45.348643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956fbe5-6c1a-4565-a06c-9b5c113cf9cf None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:45.356444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:45.419857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb07985-2989-4447-b6fc-5b26f9dac5bd None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:45.461786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fec4f2-f61b-44bf-a7a8-ceabf77697c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:45.462416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec4f2-f61b-44bf-a7a8-ceabf77697c9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:45.462780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec4f2-f61b-44bf-a7a8-ceabf77697c9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:45.463029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec4f2-f61b-44bf-a7a8-ceabf77697c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:45.470455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fec4f2-f61b-44bf-a7a8-ceabf77697c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:45.471452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956fbe5-6c1a-4565-a06c-9b5c113cf9cf None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:45.471930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fec4f2-f61b-44bf-a7a8-ceabf77697c9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-f3fec4f2-f61b-44bf-a7a8-ceabf77697c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:45.483313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:45.484355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:45.485294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:45.485294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:45.485294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:45.485294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:45.485294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:45.485294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:45.485294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:45.485294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:45.485294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 523/1074] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:45.490108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fec4f2-f61b-44bf-a7a8-ceabf77697c9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:45.511178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f15ff3-3977-477f-818d-c53421db4acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:45.512031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15ff3-3977-477f-818d-c53421db4acd None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:45.512969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15ff3-3977-477f-818d-c53421db4acd None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:45.513585 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15ff3-3977-477f-818d-c53421db4acd None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:45.514326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f15ff3-3977-477f-818d-c53421db4acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:45.514801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f15ff3-3977-477f-818d-c53421db4acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:45.516843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f15ff3-3977-477f-818d-c53421db4acd None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:45.543901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 552/1075] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:45.629342 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fec4f2-f61b-44bf-a7a8-ceabf77697c9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:45.638224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:45.668710 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e86fc-6189-4f6f-93e0-d5911cf730f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:45.669331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e86fc-6189-4f6f-93e0-d5911cf730f6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:45.669612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e86fc-6189-4f6f-93e0-d5911cf730f6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:45.669889 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e86fc-6189-4f6f-93e0-d5911cf730f6 None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:45.670864 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e86fc-6189-4f6f-93e0-d5911cf730f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:45.671176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e86fc-6189-4f6f-93e0-d5911cf730f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:45.673243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e86fc-6189-4f6f-93e0-d5911cf730f6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:45.686667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 524/1076] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:45.692672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f15ff3-3977-477f-818d-c53421db4acd None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:45.702890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:45.818240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f15ff3-3977-477f-818d-c53421db4acd None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:45.866765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c560ede-3e5d-4081-81e9-2381713a6d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:45.867705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c560ede-3e5d-4081-81e9-2381713a6d4a None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:45.868092 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c560ede-3e5d-4081-81e9-2381713a6d4a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:45.868371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c560ede-3e5d-4081-81e9-2381713a6d4a None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/d1b01c21c38949938141eefdc4939385/roles/3fec161072094f87aad16699524908bc` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:45.870486 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c560ede-3e5d-4081-81e9-2381713a6d4a None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=d1b01c21c38949938141eefdc4939385, role_id=3fec161072094f87aad16699524908bc)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:45.871975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c560ede-3e5d-4081-81e9-2381713a6d4a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:45.914169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709e9c5-0c45-40d4-9698-7fe9584c8239 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:45.914463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709e9c5-0c45-40d4-9698-7fe9584c8239 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:45.914740 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709e9c5-0c45-40d4-9698-7fe9584c8239 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:45.916121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 525/1077] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:45.921545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56971802-f2a4-4e76-9490-7a32b87a1936 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:45.922861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56971802-f2a4-4e76-9490-7a32b87a1936 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:45.922861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56971802-f2a4-4e76-9490-7a32b87a1936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:45.939685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-56971802-f2a4-4e76-9490-7a32b87a1936 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:45.940916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-56971802-f2a4-4e76-9490-7a32b87a1936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:45.972331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 553/1078] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:32:45 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/d1b01c21c38949938141eefdc4939385/roles/3fec161072094f87aad16699524908bc => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:32:46.032874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 526/1079] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:46.315259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c560ede-3e5d-4081-81e9-2381713a6d4a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:46.383973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:46.422860 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c560ede-3e5d-4081-81e9-2381713a6d4a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:46.462008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb867f9-c08d-433a-8f31-b708ff263d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:46.462731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb867f9-c08d-433a-8f31-b708ff263d1e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:46.463058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb867f9-c08d-433a-8f31-b708ff263d1e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:46.463386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb867f9-c08d-433a-8f31-b708ff263d1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:46.474485 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb867f9-c08d-433a-8f31-b708ff263d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-dfb867f9-c08d-433a-8f31-b708ff263d1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:46.479684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:46.480677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:46.481586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 554/1080] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:32:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:46.486864 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56971802-f2a4-4e76-9490-7a32b87a1936 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:46.584409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:46.590332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56971802-f2a4-4e76-9490-7a32b87a1936 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:46.624824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeda957-22f8-4372-b9a8-d33abeb23209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:46.625620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeda957-22f8-4372-b9a8-d33abeb23209 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:46.626279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeda957-22f8-4372-b9a8-d33abeb23209 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:46.626560 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeda957-22f8-4372-b9a8-d33abeb23209 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:46.627226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeda957-22f8-4372-b9a8-d33abeb23209 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:46.627533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeda957-22f8-4372-b9a8-d33abeb23209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:46.628938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeda957-22f8-4372-b9a8-d33abeb23209 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:46.638013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 527/1081] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:32:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:46.643562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb867f9-c08d-433a-8f31-b708ff263d1e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:46.743032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb867f9-c08d-433a-8f31-b708ff263d1e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:46.768636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:46.777216 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cea22a-e294-4fce-98e7-ccb47a7667d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:46.777863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cea22a-e294-4fce-98e7-ccb47a7667d3 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:46.789237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cea22a-e294-4fce-98e7-ccb47a7667d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:46.789237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cea22a-e294-4fce-98e7-ccb47a7667d3 None admin] PATH_INFO: `/v3/users/a59d38223c774a5581332246f664c37d` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:46.789237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cea22a-e294-4fce-98e7-ccb47a7667d3 None admin] RBAC: Authorizing `identity:get_user(user_id=a59d38223c774a5581332246f664c37d)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:46.789237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cea22a-e294-4fce-98e7-ccb47a7667d3 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:46.789237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 555/1082] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:32:46 2022] GET /identity/v3/users/a59d38223c774a5581332246f664c37d => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:46.789632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeda957-22f8-4372-b9a8-d33abeb23209 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:46.889152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeda957-22f8-4372-b9a8-d33abeb23209 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:46.921833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392398e6-ee10-4a13-ac01-a3e53ea91f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:46.922458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392398e6-ee10-4a13-ac01-a3e53ea91f28 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:46.922743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392398e6-ee10-4a13-ac01-a3e53ea91f28 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:46.923020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392398e6-ee10-4a13-ac01-a3e53ea91f28 None admin] PATH_INFO: `/v3/projects/512b66fecdb843789f5257338378b59a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:46.924176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392398e6-ee10-4a13-ac01-a3e53ea91f28 None admin] RBAC: Authorizing `identity:get_project(project_id=512b66fecdb843789f5257338378b59a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:46.925595 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392398e6-ee10-4a13-ac01-a3e53ea91f28 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:46.927224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 528/1083] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:32:46 2022] GET /identity/v3/projects/512b66fecdb843789f5257338378b59a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:46.933602 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cea22a-e294-4fce-98e7-ccb47a7667d3 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:47.046421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cea22a-e294-4fce-98e7-ccb47a7667d3 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:47.085165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced57f11-f500-43c0-b715-679bff548962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:47.085843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced57f11-f500-43c0-b715-679bff548962 None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:47.086131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced57f11-f500-43c0-b715-679bff548962 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:47.086410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced57f11-f500-43c0-b715-679bff548962 None admin] PATH_INFO: `/v3/projects/512b66fecdb843789f5257338378b59a/users/a59d38223c774a5581332246f664c37d/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:47.088458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced57f11-f500-43c0-b715-679bff548962 None admin] RBAC: Authorizing `identity:create_grant(project_id=512b66fecdb843789f5257338378b59a, user_id=a59d38223c774a5581332246f664c37d, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:47.089918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced57f11-f500-43c0-b715-679bff548962 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:47.163524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 556/1084] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:32:46 2022] PUT /identity/v3/projects/512b66fecdb843789f5257338378b59a/users/a59d38223c774a5581332246f664c37d/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:32:47.971628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86286c5c-f701-4fc8-b795-ff7339e41c93 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:47.972258 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86286c5c-f701-4fc8-b795-ff7339e41c93 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:47.972381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86286c5c-f701-4fc8-b795-ff7339e41c93 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:47.973732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 529/1085] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:47.979337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c435c2e-7c67-4637-9ade-78fbd40d6d0f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:47.979824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c435c2e-7c67-4637-9ade-78fbd40d6d0f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:47.980081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c435c2e-7c67-4637-9ade-78fbd40d6d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:47.999292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0c435c2e-7c67-4637-9ade-78fbd40d6d0f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:48.000542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0c435c2e-7c67-4637-9ade-78fbd40d6d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:48.051091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:48.070317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 557/1086] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:48.348491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86286c5c-f701-4fc8-b795-ff7339e41c93 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:48.379393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:48.452884 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86286c5c-f701-4fc8-b795-ff7339e41c93 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:48.486905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaa181-b1fd-49a4-ab95-c7d1cb1cbe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:48.487533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaa181-b1fd-49a4-ab95-c7d1cb1cbe8c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:48.487815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaa181-b1fd-49a4-ab95-c7d1cb1cbe8c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:48.488119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaa181-b1fd-49a4-ab95-c7d1cb1cbe8c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:48.498777 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbaa181-b1fd-49a4-ab95-c7d1cb1cbe8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-7bbaa181-b1fd-49a4-ab95-c7d1cb1cbe8c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:48.503426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:48.504481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:48.505383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 530/1087] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Aug 26 15:32:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:48.510575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c435c2e-7c67-4637-9ade-78fbd40d6d0f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:48.611795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c435c2e-7c67-4637-9ade-78fbd40d6d0f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:48.646271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac1562-58da-43ee-b52f-3ce4ebad17f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:48.646938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac1562-58da-43ee-b52f-3ce4ebad17f6 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:48.647221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac1562-58da-43ee-b52f-3ce4ebad17f6 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:48.647495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac1562-58da-43ee-b52f-3ce4ebad17f6 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:48.648259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac1562-58da-43ee-b52f-3ce4ebad17f6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:48.648564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac1562-58da-43ee-b52f-3ce4ebad17f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:48.650015 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac1562-58da-43ee-b52f-3ce4ebad17f6 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:48.659857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 558/1088] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Aug 26 15:32:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:48.665530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaa181-b1fd-49a4-ab95-c7d1cb1cbe8c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:48.749480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:48.764022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaa181-b1fd-49a4-ab95-c7d1cb1cbe8c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:48.799159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42c482-f493-4873-8e66-639e8678cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:48.799788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42c482-f493-4873-8e66-639e8678cc3c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:48.800147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42c482-f493-4873-8e66-639e8678cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:48.800419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42c482-f493-4873-8e66-639e8678cc3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:48.810679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42c482-f493-4873-8e66-639e8678cc3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:48.812182 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42c482-f493-4873-8e66-639e8678cc3c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-ff42c482-f493-4873-8e66-639e8678cc3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:48.826685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:48.834565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:48.835719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:48.835719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:48.835719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:48.835719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 531/1089] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:48.836102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac1562-58da-43ee-b52f-3ce4ebad17f6 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:48.882548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:48.935707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac1562-58da-43ee-b52f-3ce4ebad17f6 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:48.969414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c1fb0-d2dd-4aa3-8723-19853b5c1ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:48.970019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c1fb0-d2dd-4aa3-8723-19853b5c1ac5 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:48.970303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c1fb0-d2dd-4aa3-8723-19853b5c1ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:48.970584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c1fb0-d2dd-4aa3-8723-19853b5c1ac5 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:48.971241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c1fb0-d2dd-4aa3-8723-19853b5c1ac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:48.971547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c1fb0-d2dd-4aa3-8723-19853b5c1ac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:48.972960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c1fb0-d2dd-4aa3-8723-19853b5c1ac5 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:48.985407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 559/1090] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:48.991256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42c482-f493-4873-8e66-639e8678cc3c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:49.068114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f2da6-1f33-40f6-b933-cff2ae47ebe4 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:49.068769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f2da6-1f33-40f6-b933-cff2ae47ebe4 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:49.068769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f2da6-1f33-40f6-b933-cff2ae47ebe4 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:49.070018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 560/1091] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:32:49.075483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca3ed5-36be-498f-96f9-4391568fdebf None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:49.075771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca3ed5-36be-498f-96f9-4391568fdebf None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:49.076278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca3ed5-36be-498f-96f9-4391568fdebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:49.093449 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42c482-f493-4873-8e66-639e8678cc3c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:49.094526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-5eca3ed5-36be-498f-96f9-4391568fdebf None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:49.095728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5eca3ed5-36be-498f-96f9-4391568fdebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:49.126347 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66316719-c6eb-48ec-bfd5-089566bccb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:49.127085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66316719-c6eb-48ec-bfd5-089566bccb7c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:49.127374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66316719-c6eb-48ec-bfd5-089566bccb7c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:49.127973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66316719-c6eb-48ec-bfd5-089566bccb7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:49.135442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66316719-c6eb-48ec-bfd5-089566bccb7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:49.137364 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66316719-c6eb-48ec-bfd5-089566bccb7c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:49.139233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-66316719-c6eb-48ec-bfd5-089566bccb7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:32:49.149600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:49.150689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:49.152145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:49.152145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:49.152145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:32:49.152145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:32:49.152145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:32:49.152145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:32:49.152145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:32:49.152145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:32:49.152145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:32:49.152145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:49.152145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 532/1092] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Aug 26 15:32:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:49.157128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66316719-c6eb-48ec-bfd5-089566bccb7c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:49.165001 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 561/1093] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:49.255790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66316719-c6eb-48ec-bfd5-089566bccb7c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:49.291119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b3ac9-3b41-4788-b6dc-079878aba2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:49.291727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b3ac9-3b41-4788-b6dc-079878aba2ab None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:49.292114 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b3ac9-3b41-4788-b6dc-079878aba2ab None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:49.292394 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b3ac9-3b41-4788-b6dc-079878aba2ab None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:49.293120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b3ac9-3b41-4788-b6dc-079878aba2ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:49.293426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b3ac9-3b41-4788-b6dc-079878aba2ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:49.295093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b3ac9-3b41-4788-b6dc-079878aba2ab None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:49.311680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 533/1094] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Aug 26 15:32:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:49.317422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca3ed5-36be-498f-96f9-4391568fdebf None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:49.352586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:49.416255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca3ed5-36be-498f-96f9-4391568fdebf None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:49.432275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b3ac9-3b41-4788-b6dc-079878aba2ab None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:49.454151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52306c4a-f1f6-4f29-8ade-141c7bf26d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:49.454869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52306c4a-f1f6-4f29-8ade-141c7bf26d84 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:49.455230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52306c4a-f1f6-4f29-8ade-141c7bf26d84 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:49.455583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52306c4a-f1f6-4f29-8ade-141c7bf26d84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:49.493559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:49.527313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52306c4a-f1f6-4f29-8ade-141c7bf26d84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:49.529285 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52306c4a-f1f6-4f29-8ade-141c7bf26d84 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:49.539052 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b3ac9-3b41-4788-b6dc-079878aba2ab None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:49.564864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:49.576578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f72231-d6c3-463e-a13a-1b58ee70ef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:49.577217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72231-d6c3-463e-a13a-1b58ee70ef48 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:49.577517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72231-d6c3-463e-a13a-1b58ee70ef48 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:49.577783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72231-d6c3-463e-a13a-1b58ee70ef48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:49.588383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f72231-d6c3-463e-a13a-1b58ee70ef48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-f7f72231-d6c3-463e-a13a-1b58ee70ef48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:32:49.593540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:49.594732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:32:49.595629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 534/1095] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:32:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:32:49.600595 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f72231-d6c3-463e-a13a-1b58ee70ef48 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:49.638810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 562/1096] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:32:49.644751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52306c4a-f1f6-4f29-8ade-141c7bf26d84 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:49.707254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f72231-d6c3-463e-a13a-1b58ee70ef48 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:49.722810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:49.743649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c4fb1-e553-414e-b062-80405e5b0c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:49.744443 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c4fb1-e553-414e-b062-80405e5b0c9f None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:49.744775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c4fb1-e553-414e-b062-80405e5b0c9f None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:49.745113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c4fb1-e553-414e-b062-80405e5b0c9f None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:49.745695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52306c4a-f1f6-4f29-8ade-141c7bf26d84 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:49.745834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c4fb1-e553-414e-b062-80405e5b0c9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:49.746754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c4fb1-e553-414e-b062-80405e5b0c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:49.748374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c4fb1-e553-414e-b062-80405e5b0c9f None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:49.758245 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 535/1097] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:32:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:49.764684 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c4fb1-e553-414e-b062-80405e5b0c9f None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:49.780757 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697abc6-1e1c-4870-9061-6c54d99851c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:49.781386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697abc6-1e1c-4870-9061-6c54d99851c3 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:49.781678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697abc6-1e1c-4870-9061-6c54d99851c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:49.781957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697abc6-1e1c-4870-9061-6c54d99851c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:49.784653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:49.794060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697abc6-1e1c-4870-9061-6c54d99851c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:49.795819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697abc6-1e1c-4870-9061-6c54d99851c3 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-1697abc6-1e1c-4870-9061-6c54d99851c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:32:49.810345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:49.811536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:32:49.812699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:32:49.812699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:32:49.812699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:49.812699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 563/1098] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:49.817189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697abc6-1e1c-4870-9061-6c54d99851c3 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:49.881853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c4fb1-e553-414e-b062-80405e5b0c9f None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:49.921128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f352598-488d-409f-909b-f86e656f2a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:49.921493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697abc6-1e1c-4870-9061-6c54d99851c3 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:49.921817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f352598-488d-409f-909b-f86e656f2a85 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:49.922118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f352598-488d-409f-909b-f86e656f2a85 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:49.922396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f352598-488d-409f-909b-f86e656f2a85 None admin] PATH_INFO: `/v3/users/a59d38223c774a5581332246f664c37d` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:49.923637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f352598-488d-409f-909b-f86e656f2a85 None admin] RBAC: Authorizing `identity:get_user(user_id=a59d38223c774a5581332246f664c37d)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:49.925085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f352598-488d-409f-909b-f86e656f2a85 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:49.926879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 536/1099] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:32:49 2022] GET /identity/v3/users/a59d38223c774a5581332246f664c37d => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:49.932890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f352598-488d-409f-909b-f86e656f2a85 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:49.958751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cd8c5-2cf6-4a97-bf73-896701d1e412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:49.959319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cd8c5-2cf6-4a97-bf73-896701d1e412 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:49.959605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cd8c5-2cf6-4a97-bf73-896701d1e412 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:49.959889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cd8c5-2cf6-4a97-bf73-896701d1e412 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:49.960547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1cd8c5-2cf6-4a97-bf73-896701d1e412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:49.960856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1cd8c5-2cf6-4a97-bf73-896701d1e412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:49.962210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1cd8c5-2cf6-4a97-bf73-896701d1e412 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:49.974333 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 564/1100] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:49.980836 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cd8c5-2cf6-4a97-bf73-896701d1e412 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:49.983252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:50.036966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f352598-488d-409f-909b-f86e656f2a85 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:50.039638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:50.075178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd49f23-1186-4ec4-8efb-e2dbed6d23e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:50.075787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd49f23-1186-4ec4-8efb-e2dbed6d23e4 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:50.076197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd49f23-1186-4ec4-8efb-e2dbed6d23e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:50.076472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd49f23-1186-4ec4-8efb-e2dbed6d23e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:50.091603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cd8c5-2cf6-4a97-bf73-896701d1e412 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:50.126535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef5fc1-a3dd-487f-8abe-4f56fef41b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:50.127164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef5fc1-a3dd-487f-8abe-4f56fef41b89 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:50.127454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef5fc1-a3dd-487f-8abe-4f56fef41b89 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:50.127735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef5fc1-a3dd-487f-8abe-4f56fef41b89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:50.134569 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef5fc1-a3dd-487f-8abe-4f56fef41b89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:50.136123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef5fc1-a3dd-487f-8abe-4f56fef41b89 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:50.143168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd49f23-1186-4ec4-8efb-e2dbed6d23e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:50.144652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd49f23-1186-4ec4-8efb-e2dbed6d23e4 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-3aef5fc1-a3dd-487f-8abe-4f56fef41b89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:32:50.147274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:32:50.148480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:32:50.149919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:32:50.149919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:32:50.149919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:32:50.149919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:32:50.149919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:32:50.149919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:32:50.149919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:32:50.149919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:32:50.149919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 565/1101] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:32:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:32:50.154837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef5fc1-a3dd-487f-8abe-4f56fef41b89 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:50.194642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:50.248894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 537/1102] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 15:32:50.254671 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd49f23-1186-4ec4-8efb-e2dbed6d23e4 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:50.258961 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef5fc1-a3dd-487f-8abe-4f56fef41b89 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:50.268529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:50.294062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74235c6-a271-4878-acbc-11f213366c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:50.294921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74235c6-a271-4878-acbc-11f213366c42 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:50.295207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74235c6-a271-4878-acbc-11f213366c42 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:50.295482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74235c6-a271-4878-acbc-11f213366c42 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:50.296249 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74235c6-a271-4878-acbc-11f213366c42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:50.296565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74235c6-a271-4878-acbc-11f213366c42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:50.297980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74235c6-a271-4878-acbc-11f213366c42 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:50.309870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 566/1103] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:32:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:50.315800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74235c6-a271-4878-acbc-11f213366c42 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:50.358975 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd49f23-1186-4ec4-8efb-e2dbed6d23e4 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:50.394093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41031bd2-bd36-49b3-a5f8-b0f5ea9cf571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:50.394723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41031bd2-bd36-49b3-a5f8-b0f5ea9cf571 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:50.395009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41031bd2-bd36-49b3-a5f8-b0f5ea9cf571 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:50.395290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41031bd2-bd36-49b3-a5f8-b0f5ea9cf571 None admin] PATH_INFO: `/v3/projects/512b66fecdb843789f5257338378b59a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:50.396431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41031bd2-bd36-49b3-a5f8-b0f5ea9cf571 None admin] RBAC: Authorizing `identity:get_project(project_id=512b66fecdb843789f5257338378b59a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:50.397855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41031bd2-bd36-49b3-a5f8-b0f5ea9cf571 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:50.399485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 538/1104] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:32:50 2022] GET /identity/v3/projects/512b66fecdb843789f5257338378b59a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:50.406091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41031bd2-bd36-49b3-a5f8-b0f5ea9cf571 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:50.414500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74235c6-a271-4878-acbc-11f213366c42 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:50.423060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:50.450560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a306a46-a24f-447e-b86e-d240fede75bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:50.451187 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a306a46-a24f-447e-b86e-d240fede75bf None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:50.451448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a306a46-a24f-447e-b86e-d240fede75bf None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:50.451740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a306a46-a24f-447e-b86e-d240fede75bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:50.452474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a306a46-a24f-447e-b86e-d240fede75bf None admin] RBAC: Adding query filter params (role.id=3fec161072094f87aad16699524908bc, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=d1b01c21c38949938141eefdc4939385) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:50.452781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a306a46-a24f-447e-b86e-d240fede75bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:50.454285 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a306a46-a24f-447e-b86e-d240fede75bf None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:50.460233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 567/1105] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:50 2022] GET /identity/v3/role_assignments?user.id=d1b01c21c38949938141eefdc4939385&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=3fec161072094f87aad16699524908bc => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:50.514380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41031bd2-bd36-49b3-a5f8-b0f5ea9cf571 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:50.552140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996375e-4258-41c0-9428-37512627bb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:50.552756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996375e-4258-41c0-9428-37512627bb35 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:50.553040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996375e-4258-41c0-9428-37512627bb35 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:50.553318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996375e-4258-41c0-9428-37512627bb35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:50.554061 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996375e-4258-41c0-9428-37512627bb35 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=512b66fecdb843789f5257338378b59a, user.id=a59d38223c774a5581332246f664c37d) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:50.554367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996375e-4258-41c0-9428-37512627bb35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:50.555889 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996375e-4258-41c0-9428-37512627bb35 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:50.562450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 539/1106] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:50 2022] GET /identity/v3/role_assignments?user.id=a59d38223c774a5581332246f664c37d&scope.project.id=512b66fecdb843789f5257338378b59a&role.id=6ba5a5add2174983be37258aad1ee404 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:32:52.517733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564eb99-5bee-40c0-b8b9-d4a68d6b923b None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:52.518296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564eb99-5bee-40c0-b8b9-d4a68d6b923b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:52.518296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564eb99-5bee-40c0-b8b9-d4a68d6b923b None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:52.519650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 568/1107] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:52.524963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb00e0-aece-4008-8b2c-65d1dbe84995 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:52.525406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb00e0-aece-4008-8b2c-65d1dbe84995 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:52.525513 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb00e0-aece-4008-8b2c-65d1dbe84995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:52.544134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-3fdb00e0-aece-4008-8b2c-65d1dbe84995 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:52.545597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3fdb00e0-aece-4008-8b2c-65d1dbe84995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:52.612177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 540/1108] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:52.882174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564eb99-5bee-40c0-b8b9-d4a68d6b923b None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:52.913877 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:52.982714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564eb99-5bee-40c0-b8b9-d4a68d6b923b None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:53.019141 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde18009-d561-469f-9f48-da5b6f883297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:53.019756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde18009-d561-469f-9f48-da5b6f883297 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:53.020085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde18009-d561-469f-9f48-da5b6f883297 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:53.020367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde18009-d561-469f-9f48-da5b6f883297 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:53.021603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde18009-d561-469f-9f48-da5b6f883297 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:53.023067 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde18009-d561-469f-9f48-da5b6f883297 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:53.024692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 569/1109] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:32:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:53.030679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb00e0-aece-4008-8b2c-65d1dbe84995 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:53.040648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:53.127549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb00e0-aece-4008-8b2c-65d1dbe84995 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:53.160315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca594749-12f2-4188-abbe-369788a63350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:53.160850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca594749-12f2-4188-abbe-369788a63350 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:53.161134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca594749-12f2-4188-abbe-369788a63350 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:53.161416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca594749-12f2-4188-abbe-369788a63350 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:53.162056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca594749-12f2-4188-abbe-369788a63350 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:53.163496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca594749-12f2-4188-abbe-369788a63350 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:53.270007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 541/1110] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:32:53 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:32:55.256223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3d54-8883-4f5b-bc8c-cc2fab4092ce None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:55.256789 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3d54-8883-4f5b-bc8c-cc2fab4092ce None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:55.256897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3d54-8883-4f5b-bc8c-cc2fab4092ce None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:55.258387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 570/1111] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:55.265404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed2ce8-41aa-4309-ac28-b51f5ae0f16d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:55.266059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed2ce8-41aa-4309-ac28-b51f5ae0f16d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:55.266059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed2ce8-41aa-4309-ac28-b51f5ae0f16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:55.284340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-faed2ce8-41aa-4309-ac28-b51f5ae0f16d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:55.285531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-faed2ce8-41aa-4309-ac28-b51f5ae0f16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:55.343119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:55.351637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 542/1112] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:32:55.631411 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec3d54-8883-4f5b-bc8c-cc2fab4092ce None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:55.729357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:55.733749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec3d54-8883-4f5b-bc8c-cc2fab4092ce None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:55.767210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9ccab-61e2-406c-bab9-5eada7cdeee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:55.768128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9ccab-61e2-406c-bab9-5eada7cdeee0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:55.768412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9ccab-61e2-406c-bab9-5eada7cdeee0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:55.768690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9ccab-61e2-406c-bab9-5eada7cdeee0 None admin] PATH_INFO: `/v3/roles/e0822035b245457eade1325bbe888913` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:55.770072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9ccab-61e2-406c-bab9-5eada7cdeee0 None admin] RBAC: Authorizing `identity:get_role(role_id=e0822035b245457eade1325bbe888913)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:55.771472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9ccab-61e2-406c-bab9-5eada7cdeee0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:55.772810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 571/1113] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:32:55 2022] GET /identity/v3/roles/e0822035b245457eade1325bbe888913 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:55.779203 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed2ce8-41aa-4309-ac28-b51f5ae0f16d None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:55.889733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed2ce8-41aa-4309-ac28-b51f5ae0f16d None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:55.924120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed334c0-5f96-4c82-b3d1-69f01a94ab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:55.924740 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed334c0-5f96-4c82-b3d1-69f01a94ab35 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:55.925039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed334c0-5f96-4c82-b3d1-69f01a94ab35 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:55.925297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed334c0-5f96-4c82-b3d1-69f01a94ab35 None admin] PATH_INFO: `/v3/users/304cd264fc574ad49e4a6d9a0a06ecf5` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:55.938117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed334c0-5f96-4c82-b3d1-69f01a94ab35 None admin] RBAC: Authorizing `identity:get_user(user_id=304cd264fc574ad49e4a6d9a0a06ecf5)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:55.939548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed334c0-5f96-4c82-b3d1-69f01a94ab35 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:55.941160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 543/1114] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:32:55 2022] GET /identity/v3/users/304cd264fc574ad49e4a6d9a0a06ecf5 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:55.947050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9ccab-61e2-406c-bab9-5eada7cdeee0 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:56.045690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9ccab-61e2-406c-bab9-5eada7cdeee0 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:56.081409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ffb80-6e07-4ffd-a7f2-374d84c26263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:56.082007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ffb80-6e07-4ffd-a7f2-374d84c26263 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:56.082370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ffb80-6e07-4ffd-a7f2-374d84c26263 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:56.082652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ffb80-6e07-4ffd-a7f2-374d84c26263 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:56.116661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:56.150360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ffb80-6e07-4ffd-a7f2-374d84c26263 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:56.151757 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ffb80-6e07-4ffd-a7f2-374d84c26263 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:56.247179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 572/1115] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:32:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:32:56.253282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed334c0-5f96-4c82-b3d1-69f01a94ab35 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:56.309277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:56.355790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed334c0-5f96-4c82-b3d1-69f01a94ab35 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:56.387946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c12c48-3cd3-40a6-a1ad-0739434e1fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:56.388567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c12c48-3cd3-40a6-a1ad-0739434e1fa7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:56.388853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c12c48-3cd3-40a6-a1ad-0739434e1fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:56.389134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c12c48-3cd3-40a6-a1ad-0739434e1fa7 None admin] PATH_INFO: `/v3/projects/512b66fecdb843789f5257338378b59a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:56.390251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c12c48-3cd3-40a6-a1ad-0739434e1fa7 None admin] RBAC: Authorizing `identity:get_project(project_id=512b66fecdb843789f5257338378b59a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:56.391676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c12c48-3cd3-40a6-a1ad-0739434e1fa7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:56.393321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 544/1116] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:32:56 2022] GET /identity/v3/projects/512b66fecdb843789f5257338378b59a => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:56.399882 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ffb80-6e07-4ffd-a7f2-374d84c26263 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:56.482861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:56.520551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ffb80-6e07-4ffd-a7f2-374d84c26263 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:56.553829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7d468-0480-4440-8008-a4acba8616c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:56.554424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7d468-0480-4440-8008-a4acba8616c8 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:56.554709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7d468-0480-4440-8008-a4acba8616c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:56.554989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7d468-0480-4440-8008-a4acba8616c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:56.555713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7d468-0480-4440-8008-a4acba8616c8 None admin] RBAC: Adding query filter params (role.id=e0822035b245457eade1325bbe888913, scope.project.id=512b66fecdb843789f5257338378b59a, user.id=304cd264fc574ad49e4a6d9a0a06ecf5) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:32:56.556387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7d468-0480-4440-8008-a4acba8616c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:56.558388 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7d468-0480-4440-8008-a4acba8616c8 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:56.563590 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 573/1117] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:32:56 2022] GET /identity/v3/role_assignments?user.id=304cd264fc574ad49e4a6d9a0a06ecf5&scope.project.id=512b66fecdb843789f5257338378b59a&role.id=e0822035b245457eade1325bbe888913 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:58.469274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1dba6-f17c-48fb-a4a5-f6ef3cb345f9 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:58.469834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1dba6-f17c-48fb-a4a5-f6ef3cb345f9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:58.469834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1dba6-f17c-48fb-a4a5-f6ef3cb345f9 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:58.471183 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 545/1118] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:32:58.477196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9770d-7ca7-469f-b0ab-28891ff0241b None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:58.477638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9770d-7ca7-469f-b0ab-28891ff0241b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:58.477741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9770d-7ca7-469f-b0ab-28891ff0241b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:58.495895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3cf9770d-7ca7-469f-b0ab-28891ff0241b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:32:58.497105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3cf9770d-7ca7-469f-b0ab-28891ff0241b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:32:58.564283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 574/1119] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:32:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:32:58.836093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1dba6-f17c-48fb-a4a5-f6ef3cb345f9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:58.939085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1dba6-f17c-48fb-a4a5-f6ef3cb345f9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:58.958549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:58.971959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe06b5-cb2f-4012-803a-ce62d27800de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:58.972584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe06b5-cb2f-4012-803a-ce62d27800de None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:58.972871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe06b5-cb2f-4012-803a-ce62d27800de None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:58.973152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe06b5-cb2f-4012-803a-ce62d27800de None admin] PATH_INFO: `/v3/roles/e0822035b245457eade1325bbe888913` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:58.974548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe06b5-cb2f-4012-803a-ce62d27800de None admin] RBAC: Authorizing `identity:get_role(role_id=e0822035b245457eade1325bbe888913)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:58.975952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe06b5-cb2f-4012-803a-ce62d27800de None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:58.977261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 546/1120] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:32:58 2022] GET /identity/v3/roles/e0822035b245457eade1325bbe888913 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:58.983555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9770d-7ca7-469f-b0ab-28891ff0241b None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:59.043606 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:59.081895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9770d-7ca7-469f-b0ab-28891ff0241b None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:59.114153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fe6c7-bbb3-40f5-8ae3-8ffe9d54a72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:59.114710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fe6c7-bbb3-40f5-8ae3-8ffe9d54a72d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:59.114992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fe6c7-bbb3-40f5-8ae3-8ffe9d54a72d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:59.115269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fe6c7-bbb3-40f5-8ae3-8ffe9d54a72d None admin] PATH_INFO: `/v3/users/304cd264fc574ad49e4a6d9a0a06ecf5` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:59.116433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709fe6c7-bbb3-40f5-8ae3-8ffe9d54a72d None admin] RBAC: Authorizing `identity:get_user(user_id=304cd264fc574ad49e4a6d9a0a06ecf5)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:59.117864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709fe6c7-bbb3-40f5-8ae3-8ffe9d54a72d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:59.119418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 575/1121] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:32:58 2022] GET /identity/v3/users/304cd264fc574ad49e4a6d9a0a06ecf5 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:59.125874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe06b5-cb2f-4012-803a-ce62d27800de None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:59.223697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe06b5-cb2f-4012-803a-ce62d27800de None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:59.256270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6676c-9a30-4733-92dd-887696f3d92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:59.256891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6676c-9a30-4733-92dd-887696f3d92e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:59.257177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6676c-9a30-4733-92dd-887696f3d92e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:59.257455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6676c-9a30-4733-92dd-887696f3d92e None admin] PATH_INFO: `/v3/projects/512b66fecdb843789f5257338378b59a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:59.258569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6676c-9a30-4733-92dd-887696f3d92e None admin] RBAC: Authorizing `identity:get_project(project_id=512b66fecdb843789f5257338378b59a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:59.260025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6676c-9a30-4733-92dd-887696f3d92e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:59.261674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 547/1122] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:32:59 2022] GET /identity/v3/projects/512b66fecdb843789f5257338378b59a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:32:59.267638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fe6c7-bbb3-40f5-8ae3-8ffe9d54a72d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:32:59.366442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fe6c7-bbb3-40f5-8ae3-8ffe9d54a72d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:32:59.393820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:32:59.399368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d35d8-fc53-4c7f-87c2-4f4e12dc201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:32:59.400051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d35d8-fc53-4c7f-87c2-4f4e12dc201a None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:32:59.400336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d35d8-fc53-4c7f-87c2-4f4e12dc201a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:32:59.400613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d35d8-fc53-4c7f-87c2-4f4e12dc201a None admin] PATH_INFO: `/v3/projects/512b66fecdb843789f5257338378b59a/users/304cd264fc574ad49e4a6d9a0a06ecf5/roles/e0822035b245457eade1325bbe888913` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:32:59.403434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d35d8-fc53-4c7f-87c2-4f4e12dc201a None admin] RBAC: Authorizing `identity:create_grant(project_id=512b66fecdb843789f5257338378b59a, user_id=304cd264fc574ad49e4a6d9a0a06ecf5, role_id=e0822035b245457eade1325bbe888913)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:32:59.404894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d35d8-fc53-4c7f-87c2-4f4e12dc201a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:32:59.465207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 576/1123] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:32:59 2022] PUT /identity/v3/projects/512b66fecdb843789f5257338378b59a/users/304cd264fc574ad49e4a6d9a0a06ecf5/roles/e0822035b245457eade1325bbe888913 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:33:01.428199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275f1c2-3bc7-49bb-8476-62aea90dc2b9 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:01.428749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275f1c2-3bc7-49bb-8476-62aea90dc2b9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:01.428749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275f1c2-3bc7-49bb-8476-62aea90dc2b9 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:01.430091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 548/1124] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:33:01.435931 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b0419-8026-415a-a9f9-766f5d7b8558 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:01.436314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b0419-8026-415a-a9f9-766f5d7b8558 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:01.436488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b0419-8026-415a-a9f9-766f5d7b8558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:01.455021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-267b0419-8026-415a-a9f9-766f5d7b8558 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:01.456319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-267b0419-8026-415a-a9f9-766f5d7b8558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:01.526786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 577/1125] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:01.794815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275f1c2-3bc7-49bb-8476-62aea90dc2b9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:01.851202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:01.896191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275f1c2-3bc7-49bb-8476-62aea90dc2b9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:01.929559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b19d57-ce6d-47c3-8406-951decc61a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:01.930186 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b19d57-ce6d-47c3-8406-951decc61a00 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:01.930472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b19d57-ce6d-47c3-8406-951decc61a00 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:01.930757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b19d57-ce6d-47c3-8406-951decc61a00 None admin] PATH_INFO: `/v3/roles/e0822035b245457eade1325bbe888913` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:01.932798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b19d57-ce6d-47c3-8406-951decc61a00 None admin] RBAC: Authorizing `identity:get_role(role_id=e0822035b245457eade1325bbe888913)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:01.934167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b19d57-ce6d-47c3-8406-951decc61a00 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:01.935486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 549/1126] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:33:01 2022] GET /identity/v3/roles/e0822035b245457eade1325bbe888913 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:01.941315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b0419-8026-415a-a9f9-766f5d7b8558 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:02.050873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b0419-8026-415a-a9f9-766f5d7b8558 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:02.072517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:02.087326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a43bf0-fda9-46f3-8597-33ad825c74eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:02.087992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a43bf0-fda9-46f3-8597-33ad825c74eb None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:02.088304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a43bf0-fda9-46f3-8597-33ad825c74eb None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:02.088579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a43bf0-fda9-46f3-8597-33ad825c74eb None admin] PATH_INFO: `/v3/users/304cd264fc574ad49e4a6d9a0a06ecf5` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:02.089706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a43bf0-fda9-46f3-8597-33ad825c74eb None admin] RBAC: Authorizing `identity:get_user(user_id=304cd264fc574ad49e4a6d9a0a06ecf5)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:02.091150 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a43bf0-fda9-46f3-8597-33ad825c74eb None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:02.093076 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 578/1127] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:33:01 2022] GET /identity/v3/users/304cd264fc574ad49e4a6d9a0a06ecf5 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:02.098726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b19d57-ce6d-47c3-8406-951decc61a00 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:02.206784 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b19d57-ce6d-47c3-8406-951decc61a00 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:02.230274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:02.240798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a1285-3df6-40ca-8073-cd53f20c5b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:02.241399 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a1285-3df6-40ca-8073-cd53f20c5b20 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:02.241706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a1285-3df6-40ca-8073-cd53f20c5b20 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:02.246543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a1285-3df6-40ca-8073-cd53f20c5b20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:02.307741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a1285-3df6-40ca-8073-cd53f20c5b20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:02.309151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a1285-3df6-40ca-8073-cd53f20c5b20 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:02.403357 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 550/1128] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:33:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 15:33:02.409605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a43bf0-fda9-46f3-8597-33ad825c74eb None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:02.519074 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a43bf0-fda9-46f3-8597-33ad825c74eb None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:02.551815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3644b-3952-47c2-9d5b-8977232ada34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:02.552455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3644b-3952-47c2-9d5b-8977232ada34 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:02.552740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3644b-3952-47c2-9d5b-8977232ada34 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:02.553019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3644b-3952-47c2-9d5b-8977232ada34 None admin] PATH_INFO: `/v3/projects/512b66fecdb843789f5257338378b59a` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:02.554119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3644b-3952-47c2-9d5b-8977232ada34 None admin] RBAC: Authorizing `identity:get_project(project_id=512b66fecdb843789f5257338378b59a)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:02.555556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3644b-3952-47c2-9d5b-8977232ada34 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:02.557195 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 579/1129] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:33:02 2022] GET /identity/v3/projects/512b66fecdb843789f5257338378b59a => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:02.563498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a1285-3df6-40ca-8073-cd53f20c5b20 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:02.587721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:02.661850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a1285-3df6-40ca-8073-cd53f20c5b20 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:02.695282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0ab61-2be7-4bf9-aa36-df1b03061115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:02.695901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ab61-2be7-4bf9-aa36-df1b03061115 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:02.696180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ab61-2be7-4bf9-aa36-df1b03061115 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:02.696456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ab61-2be7-4bf9-aa36-df1b03061115 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:02.697172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0ab61-2be7-4bf9-aa36-df1b03061115 None admin] RBAC: Adding query filter params (role.id=e0822035b245457eade1325bbe888913, scope.project.id=512b66fecdb843789f5257338378b59a, user.id=304cd264fc574ad49e4a6d9a0a06ecf5) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:02.698050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0ab61-2be7-4bf9-aa36-df1b03061115 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:02.699485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0ab61-2be7-4bf9-aa36-df1b03061115 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:02.705154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 551/1130] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:33:02 2022] GET /identity/v3/role_assignments?user.id=304cd264fc574ad49e4a6d9a0a06ecf5&scope.project.id=512b66fecdb843789f5257338378b59a&role.id=e0822035b245457eade1325bbe888913 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:04.649163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6deaa5-0bb1-49af-8121-96a48e96bc39 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:04.649625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6deaa5-0bb1-49af-8121-96a48e96bc39 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:04.649731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6deaa5-0bb1-49af-8121-96a48e96bc39 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:04.651112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 580/1131] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:04.656468 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbf6cb-3222-4760-84a1-14f94c7b92dd None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:04.656898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbf6cb-3222-4760-84a1-14f94c7b92dd None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:04.656990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbf6cb-3222-4760-84a1-14f94c7b92dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:04.674626 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-80fbf6cb-3222-4760-84a1-14f94c7b92dd None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:04.675826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-80fbf6cb-3222-4760-84a1-14f94c7b92dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:04.741741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 552/1132] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:05.008922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6deaa5-0bb1-49af-8121-96a48e96bc39 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:05.064780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:05.108328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6deaa5-0bb1-49af-8121-96a48e96bc39 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:05.140762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfdaab-052f-4276-9d67-417ea6f82340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:05.141374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfdaab-052f-4276-9d67-417ea6f82340 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:05.141660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfdaab-052f-4276-9d67-417ea6f82340 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:05.141939 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfdaab-052f-4276-9d67-417ea6f82340 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:05.143046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfdaab-052f-4276-9d67-417ea6f82340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:05.144508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfdaab-052f-4276-9d67-417ea6f82340 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:05.146053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 581/1133] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:33:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:05.152657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbf6cb-3222-4760-84a1-14f94c7b92dd None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:05.171303 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:05.247767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbf6cb-3222-4760-84a1-14f94c7b92dd None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:05.280376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb1d66-5a84-4697-bcb5-b0f2d74927aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:05.280915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb1d66-5a84-4697-bcb5-b0f2d74927aa None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:05.281256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb1d66-5a84-4697-bcb5-b0f2d74927aa None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:05.281623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb1d66-5a84-4697-bcb5-b0f2d74927aa None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:05.282255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb1d66-5a84-4697-bcb5-b0f2d74927aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:05.283669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb1d66-5a84-4697-bcb5-b0f2d74927aa None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:05.374153 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 553/1134] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:33:05 2022] POST /identity/v3/projects => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:33:07.286982 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d31edc-7cc7-4a70-bf7c-eae4422bd6b4 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:07.287509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d31edc-7cc7-4a70-bf7c-eae4422bd6b4 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:07.287509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d31edc-7cc7-4a70-bf7c-eae4422bd6b4 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:07.288894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 582/1135] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:07.294920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db09d2-0f29-41fe-9a5e-24cc4ff6fd74 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:07.295363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db09d2-0f29-41fe-9a5e-24cc4ff6fd74 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:07.295456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db09d2-0f29-41fe-9a5e-24cc4ff6fd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:07.314045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-f4db09d2-0f29-41fe-9a5e-24cc4ff6fd74 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:07.315235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f4db09d2-0f29-41fe-9a5e-24cc4ff6fd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:07.375031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:07.379441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 554/1136] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:07.647812 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d31edc-7cc7-4a70-bf7c-eae4422bd6b4 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:07.754820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d31edc-7cc7-4a70-bf7c-eae4422bd6b4 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:07.777454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:07.787695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3780a1-e50d-49ca-889a-552057f8dce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:07.792508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3780a1-e50d-49ca-889a-552057f8dce7 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:07.792508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3780a1-e50d-49ca-889a-552057f8dce7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:07.792508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3780a1-e50d-49ca-889a-552057f8dce7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:07.792508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3780a1-e50d-49ca-889a-552057f8dce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:07.792508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3780a1-e50d-49ca-889a-552057f8dce7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:07.793769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 583/1137] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:33:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:07.799518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db09d2-0f29-41fe-9a5e-24cc4ff6fd74 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:07.895917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db09d2-0f29-41fe-9a5e-24cc4ff6fd74 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:07.929578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1d9c4-eec4-4b61-8621-8307ab8fce45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:07.930180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1d9c4-eec4-4b61-8621-8307ab8fce45 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:07.930465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1d9c4-eec4-4b61-8621-8307ab8fce45 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:07.930745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1d9c4-eec4-4b61-8621-8307ab8fce45 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:07.931425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1d9c4-eec4-4b61-8621-8307ab8fce45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:07.933374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1d9c4-eec4-4b61-8621-8307ab8fce45 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:08.047223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 555/1138] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:33:07 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:33:09.971017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c19e0-c5d5-4342-bc02-dd10f28c8005 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:09.971548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c19e0-c5d5-4342-bc02-dd10f28c8005 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:09.971640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c19e0-c5d5-4342-bc02-dd10f28c8005 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:09.973122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 584/1139] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:09.979002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2737-3d81-4ca1-a19a-0263f16a366d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:09.979416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2737-3d81-4ca1-a19a-0263f16a366d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:09.979556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2737-3d81-4ca1-a19a-0263f16a366d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:09.998154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-084c2737-3d81-4ca1-a19a-0263f16a366d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:09.999359 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-084c2737-3d81-4ca1-a19a-0263f16a366d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:10.056119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:10.077024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 556/1140] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:10.344670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c19e0-c5d5-4342-bc02-dd10f28c8005 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:10.444050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c19e0-c5d5-4342-bc02-dd10f28c8005 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:10.478780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0efb57-1482-449e-bb93-55d4387f6d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:10.479402 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0efb57-1482-449e-bb93-55d4387f6d85 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:10.479733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0efb57-1482-449e-bb93-55d4387f6d85 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:10.480047 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0efb57-1482-449e-bb93-55d4387f6d85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:10.490112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0efb57-1482-449e-bb93-55d4387f6d85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-2f0efb57-1482-449e-bb93-55d4387f6d85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:33:10.494943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:33:10.496100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:33:10.497027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 585/1141] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:33:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:33:10.502337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c2737-3d81-4ca1-a19a-0263f16a366d None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:10.602080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c2737-3d81-4ca1-a19a-0263f16a366d None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:10.637031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bf979-a5b1-4e81-9876-a82278ceabac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:10.637628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bf979-a5b1-4e81-9876-a82278ceabac None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:10.637910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bf979-a5b1-4e81-9876-a82278ceabac None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:10.638185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bf979-a5b1-4e81-9876-a82278ceabac None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:10.638844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bf979-a5b1-4e81-9876-a82278ceabac None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:10.639147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bf979-a5b1-4e81-9876-a82278ceabac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:10.640540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bf979-a5b1-4e81-9876-a82278ceabac None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:10.648529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 557/1142] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:33:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:10.655014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0efb57-1482-449e-bb93-55d4387f6d85 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:10.707737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:10.753882 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0efb57-1482-449e-bb93-55d4387f6d85 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:10.786386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc9d8c-fa6d-48a2-b0e7-0c695ab19a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:10.787002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc9d8c-fa6d-48a2-b0e7-0c695ab19a4b None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:10.787286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc9d8c-fa6d-48a2-b0e7-0c695ab19a4b None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:10.787562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc9d8c-fa6d-48a2-b0e7-0c695ab19a4b None admin] PATH_INFO: `/v3/users/aea9b07c91724008833b2185c50155d3` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:10.801054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc9d8c-fa6d-48a2-b0e7-0c695ab19a4b None admin] RBAC: Authorizing `identity:get_user(user_id=aea9b07c91724008833b2185c50155d3)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:10.802489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc9d8c-fa6d-48a2-b0e7-0c695ab19a4b None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:10.804112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 586/1143] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:33:10 2022] GET /identity/v3/users/aea9b07c91724008833b2185c50155d3 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:10.810285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bf979-a5b1-4e81-9876-a82278ceabac None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:10.858500 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:10.908428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bf979-a5b1-4e81-9876-a82278ceabac None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:10.940592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af0fc2-27c4-4e28-8030-c4a3342606b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:10.941194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0fc2-27c4-4e28-8030-c4a3342606b1 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:10.941476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0fc2-27c4-4e28-8030-c4a3342606b1 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:10.941750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0fc2-27c4-4e28-8030-c4a3342606b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:11.005807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af0fc2-27c4-4e28-8030-c4a3342606b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:11.007255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af0fc2-27c4-4e28-8030-c4a3342606b1 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:11.052887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:11.101751 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 558/1144] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:33:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:33:11.108948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc9d8c-fa6d-48a2-b0e7-0c695ab19a4b None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:11.220269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc9d8c-fa6d-48a2-b0e7-0c695ab19a4b None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:11.231057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:11.253113 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bbc1f-d599-4f38-9858-f313b86d6f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:11.253724 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bbc1f-d599-4f38-9858-f313b86d6f52 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:11.254003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bbc1f-d599-4f38-9858-f313b86d6f52 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:11.254275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bbc1f-d599-4f38-9858-f313b86d6f52 None admin] PATH_INFO: `/v3/projects/d659a144a896463298808fbe68a67783` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:11.255378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bbc1f-d599-4f38-9858-f313b86d6f52 None admin] RBAC: Authorizing `identity:get_project(project_id=d659a144a896463298808fbe68a67783)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:11.256819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bbc1f-d599-4f38-9858-f313b86d6f52 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:11.258414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 587/1145] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:33:11 2022] GET /identity/v3/projects/d659a144a896463298808fbe68a67783 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:11.264863 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af0fc2-27c4-4e28-8030-c4a3342606b1 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:11.360395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af0fc2-27c4-4e28-8030-c4a3342606b1 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:11.392555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4ab1d-6a13-428b-be62-dd878e06146c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:11.393152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4ab1d-6a13-428b-be62-dd878e06146c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:11.393436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4ab1d-6a13-428b-be62-dd878e06146c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:11.393711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4ab1d-6a13-428b-be62-dd878e06146c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:11.394427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4ab1d-6a13-428b-be62-dd878e06146c None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=d659a144a896463298808fbe68a67783, user.id=aea9b07c91724008833b2185c50155d3) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:11.394735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4ab1d-6a13-428b-be62-dd878e06146c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:11.396179 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4ab1d-6a13-428b-be62-dd878e06146c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:11.402080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 559/1146] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:33:11 2022] GET /identity/v3/role_assignments?user.id=aea9b07c91724008833b2185c50155d3&scope.project.id=d659a144a896463298808fbe68a67783&role.id=6ba5a5add2174983be37258aad1ee404 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:13.278013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef670-b165-4d6a-ab06-ce8bfca69c2e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:13.278577 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef670-b165-4d6a-ab06-ce8bfca69c2e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:13.278577 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef670-b165-4d6a-ab06-ce8bfca69c2e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:13.279912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 588/1147] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:13.285180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290a705-f182-463c-be55-d7a3ce3b3002 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:13.285557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290a705-f182-463c-be55-d7a3ce3b3002 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:13.285744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290a705-f182-463c-be55-d7a3ce3b3002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:13.304102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-9290a705-f182-463c-be55-d7a3ce3b3002 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:13.305290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9290a705-f182-463c-be55-d7a3ce3b3002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:13.313111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:13.370594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 560/1148] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:13.641805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eef670-b165-4d6a-ab06-ce8bfca69c2e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:13.741675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eef670-b165-4d6a-ab06-ce8bfca69c2e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:13.775063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38598110-6666-4296-a29d-6d6c356a9976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:13.775682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38598110-6666-4296-a29d-6d6c356a9976 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:13.776000 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38598110-6666-4296-a29d-6d6c356a9976 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:13.776278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38598110-6666-4296-a29d-6d6c356a9976 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:13.786169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38598110-6666-4296-a29d-6d6c356a9976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-38598110-6666-4296-a29d-6d6c356a9976 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:33:13.791310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:33:13.792521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:33:13.794009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 589/1149] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:33:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:33:13.799128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290a705-f182-463c-be55-d7a3ce3b3002 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:13.894094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290a705-f182-463c-be55-d7a3ce3b3002 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:13.926591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b194a-4246-423d-8ba4-961f2e8d5ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:13.927188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b194a-4246-423d-8ba4-961f2e8d5ea3 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:13.927469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b194a-4246-423d-8ba4-961f2e8d5ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:13.927748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b194a-4246-423d-8ba4-961f2e8d5ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:13.928426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b194a-4246-423d-8ba4-961f2e8d5ea3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:13.928728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b194a-4246-423d-8ba4-961f2e8d5ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:13.930082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b194a-4246-423d-8ba4-961f2e8d5ea3 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:13.938349 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 561/1150] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:33:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:33:13.944217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38598110-6666-4296-a29d-6d6c356a9976 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:13.983283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:14.042091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38598110-6666-4296-a29d-6d6c356a9976 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:14.075168 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c518aff-b458-4135-ae29-8f2eab188177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:14.075788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c518aff-b458-4135-ae29-8f2eab188177 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:14.076101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c518aff-b458-4135-ae29-8f2eab188177 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:14.076380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c518aff-b458-4135-ae29-8f2eab188177 None admin] PATH_INFO: `/v3/users/aea9b07c91724008833b2185c50155d3` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:14.077611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c518aff-b458-4135-ae29-8f2eab188177 None admin] RBAC: Authorizing `identity:get_user(user_id=aea9b07c91724008833b2185c50155d3)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:14.079044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c518aff-b458-4135-ae29-8f2eab188177 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:14.080645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 590/1151] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:33:13 2022] GET /identity/v3/users/aea9b07c91724008833b2185c50155d3 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:14.086545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b194a-4246-423d-8ba4-961f2e8d5ea3 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:14.090555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:14.186821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b194a-4246-423d-8ba4-961f2e8d5ea3 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:14.219146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8f006-a88b-47ad-abe1-7b647cbdd921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:14.219774 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8f006-a88b-47ad-abe1-7b647cbdd921 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:14.220156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8f006-a88b-47ad-abe1-7b647cbdd921 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:14.220436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8f006-a88b-47ad-abe1-7b647cbdd921 None admin] PATH_INFO: `/v3/projects/d659a144a896463298808fbe68a67783` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:14.221549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8f006-a88b-47ad-abe1-7b647cbdd921 None admin] RBAC: Authorizing `identity:get_project(project_id=d659a144a896463298808fbe68a67783)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:14.222978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8f006-a88b-47ad-abe1-7b647cbdd921 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:14.224640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 562/1152] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:33:14 2022] GET /identity/v3/projects/d659a144a896463298808fbe68a67783 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:14.231438 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c518aff-b458-4135-ae29-8f2eab188177 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:14.328132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c518aff-b458-4135-ae29-8f2eab188177 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:14.334947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:14.361502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae9260-8915-4924-86a5-c51c583033ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:14.362156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae9260-8915-4924-86a5-c51c583033ca None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:14.362441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae9260-8915-4924-86a5-c51c583033ca None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:14.362719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae9260-8915-4924-86a5-c51c583033ca None admin] PATH_INFO: `/v3/projects/d659a144a896463298808fbe68a67783/users/aea9b07c91724008833b2185c50155d3/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:14.364792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae9260-8915-4924-86a5-c51c583033ca None admin] RBAC: Authorizing `identity:create_grant(project_id=d659a144a896463298808fbe68a67783, user_id=aea9b07c91724008833b2185c50155d3, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:14.366671 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae9260-8915-4924-86a5-c51c583033ca None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:14.451278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 591/1153] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:33:14 2022] PUT /identity/v3/projects/d659a144a896463298808fbe68a67783/users/aea9b07c91724008833b2185c50155d3/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:33:16.409665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf779f9-0f96-4eef-90ae-3174d9a0abf3 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:16.410192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf779f9-0f96-4eef-90ae-3174d9a0abf3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:16.410192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf779f9-0f96-4eef-90ae-3174d9a0abf3 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:16.411544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 563/1154] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:16.416993 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e0008-150a-4a4a-83ea-ec79e597ce3c None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:16.417419 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e0008-150a-4a4a-83ea-ec79e597ce3c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:16.417555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e0008-150a-4a4a-83ea-ec79e597ce3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:16.435885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-5e8e0008-150a-4a4a-83ea-ec79e597ce3c None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:16.437079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5e8e0008-150a-4a4a-83ea-ec79e597ce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:16.503258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 592/1155] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:16.768737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf779f9-0f96-4eef-90ae-3174d9a0abf3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:16.839296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:16.868822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf779f9-0f96-4eef-90ae-3174d9a0abf3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:16.901948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91814c5d-ad02-4e1e-9553-2285fa35bd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:16.902566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91814c5d-ad02-4e1e-9553-2285fa35bd15 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:16.902851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91814c5d-ad02-4e1e-9553-2285fa35bd15 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:16.903150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91814c5d-ad02-4e1e-9553-2285fa35bd15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:16.913882 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91814c5d-ad02-4e1e-9553-2285fa35bd15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-91814c5d-ad02-4e1e-9553-2285fa35bd15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:33:16.918808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:33:16.919922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:33:16.921005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 564/1156] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:33:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:33:16.926118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e0008-150a-4a4a-83ea-ec79e597ce3c None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:17.016029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:17.021697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e0008-150a-4a4a-83ea-ec79e597ce3c None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:17.055325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c354f79-d248-4d77-873a-a393bfe83ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:17.055956 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354f79-d248-4d77-873a-a393bfe83ec2 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:17.056244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354f79-d248-4d77-873a-a393bfe83ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:17.056523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354f79-d248-4d77-873a-a393bfe83ec2 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:17.057187 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c354f79-d248-4d77-873a-a393bfe83ec2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:17.057494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c354f79-d248-4d77-873a-a393bfe83ec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:17.058860 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c354f79-d248-4d77-873a-a393bfe83ec2 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:17.066705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 593/1157] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:33:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:17.073215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91814c5d-ad02-4e1e-9553-2285fa35bd15 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:17.169017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91814c5d-ad02-4e1e-9553-2285fa35bd15 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:17.197244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:17.202179 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9531e02-72e7-4638-aeb1-41a58db3bd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:17.202816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531e02-72e7-4638-aeb1-41a58db3bd00 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:17.203096 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531e02-72e7-4638-aeb1-41a58db3bd00 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:17.203370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531e02-72e7-4638-aeb1-41a58db3bd00 None admin] PATH_INFO: `/v3/users/aea9b07c91724008833b2185c50155d3` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:17.204493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9531e02-72e7-4638-aeb1-41a58db3bd00 None admin] RBAC: Authorizing `identity:get_user(user_id=aea9b07c91724008833b2185c50155d3)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:17.205910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9531e02-72e7-4638-aeb1-41a58db3bd00 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:17.207466 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 565/1158] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:33:17 2022] GET /identity/v3/users/aea9b07c91724008833b2185c50155d3 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:17.214433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c354f79-d248-4d77-873a-a393bfe83ec2 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:17.310126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c354f79-d248-4d77-873a-a393bfe83ec2 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:17.342290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc153dad-b10b-4a85-a380-80e424b06351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:17.342886 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc153dad-b10b-4a85-a380-80e424b06351 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:17.343166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc153dad-b10b-4a85-a380-80e424b06351 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:17.343439 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc153dad-b10b-4a85-a380-80e424b06351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:17.369232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:17.408228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc153dad-b10b-4a85-a380-80e424b06351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:17.409717 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc153dad-b10b-4a85-a380-80e424b06351 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:17.503675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 594/1159] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:33:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:33:17.510280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9531e02-72e7-4638-aeb1-41a58db3bd00 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:17.624980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9531e02-72e7-4638-aeb1-41a58db3bd00 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:17.657839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af18b7-5d65-4904-9ae1-026bdc186d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:17.658464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af18b7-5d65-4904-9ae1-026bdc186d8f None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:17.658747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af18b7-5d65-4904-9ae1-026bdc186d8f None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:17.659024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af18b7-5d65-4904-9ae1-026bdc186d8f None admin] PATH_INFO: `/v3/projects/d659a144a896463298808fbe68a67783` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:17.660154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af18b7-5d65-4904-9ae1-026bdc186d8f None admin] RBAC: Authorizing `identity:get_project(project_id=d659a144a896463298808fbe68a67783)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:17.661582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af18b7-5d65-4904-9ae1-026bdc186d8f None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:17.663219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 566/1160] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:33:17 2022] GET /identity/v3/projects/d659a144a896463298808fbe68a67783 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:17.669809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc153dad-b10b-4a85-a380-80e424b06351 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:17.725477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:17.765803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc153dad-b10b-4a85-a380-80e424b06351 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:17.798884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b54e33-7c5f-4b89-ac42-aafbd362a2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:17.799469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54e33-7c5f-4b89-ac42-aafbd362a2f9 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:17.799748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54e33-7c5f-4b89-ac42-aafbd362a2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:17.800051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54e33-7c5f-4b89-ac42-aafbd362a2f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:17.800764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b54e33-7c5f-4b89-ac42-aafbd362a2f9 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=d659a144a896463298808fbe68a67783, user.id=aea9b07c91724008833b2185c50155d3) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:17.801069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b54e33-7c5f-4b89-ac42-aafbd362a2f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:17.802479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b54e33-7c5f-4b89-ac42-aafbd362a2f9 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:17.808000 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 595/1161] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:33:17 2022] GET /identity/v3/role_assignments?user.id=aea9b07c91724008833b2185c50155d3&scope.project.id=d659a144a896463298808fbe68a67783&role.id=6ba5a5add2174983be37258aad1ee404 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:19.732979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664d917-b72f-4e2f-8ee5-287196573f15 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:19.733493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664d917-b72f-4e2f-8ee5-287196573f15 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:19.733583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664d917-b72f-4e2f-8ee5-287196573f15 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:19.734852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 567/1162] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:19.740107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3615a1e-72b9-4071-9bcf-a416def1b053 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:19.740501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3615a1e-72b9-4071-9bcf-a416def1b053 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:19.740690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3615a1e-72b9-4071-9bcf-a416def1b053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:19.758867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f3615a1e-72b9-4071-9bcf-a416def1b053 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:19.760086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f3615a1e-72b9-4071-9bcf-a416def1b053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:19.825062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 596/1163] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:20.113314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664d917-b72f-4e2f-8ee5-287196573f15 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:20.179591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:20.213465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664d917-b72f-4e2f-8ee5-287196573f15 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:20.246130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45d9ac-0461-49bd-8e3a-1cea1ecc75e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:20.246727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45d9ac-0461-49bd-8e3a-1cea1ecc75e6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:20.247017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45d9ac-0461-49bd-8e3a-1cea1ecc75e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:20.247293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45d9ac-0461-49bd-8e3a-1cea1ecc75e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:20.312413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45d9ac-0461-49bd-8e3a-1cea1ecc75e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:20.313857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45d9ac-0461-49bd-8e3a-1cea1ecc75e6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:20.375328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:20.408524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 568/1164] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:33:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:33:20.414519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3615a1e-72b9-4071-9bcf-a416def1b053 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:20.465694 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:20.510958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3615a1e-72b9-4071-9bcf-a416def1b053 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:20.543406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf53771-c516-4122-ad9c-2ebb279b3768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:20.544049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf53771-c516-4122-ad9c-2ebb279b3768 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:20.544336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf53771-c516-4122-ad9c-2ebb279b3768 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:20.544614 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf53771-c516-4122-ad9c-2ebb279b3768 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:20.551049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf53771-c516-4122-ad9c-2ebb279b3768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:20.552526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf53771-c516-4122-ad9c-2ebb279b3768 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-bbf53771-c516-4122-ad9c-2ebb279b3768 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:33:20.563604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:33:20.564883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:33:20.565784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 597/1165] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Aug 26 15:33:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:33:20.570534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45d9ac-0461-49bd-8e3a-1cea1ecc75e6 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:20.669270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45d9ac-0461-49bd-8e3a-1cea1ecc75e6 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:20.702262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48c244-b563-4a32-bd95-4dd4685bcbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:20.702859 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48c244-b563-4a32-bd95-4dd4685bcbe7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:20.703144 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48c244-b563-4a32-bd95-4dd4685bcbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:20.703423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48c244-b563-4a32-bd95-4dd4685bcbe7 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:20.704087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48c244-b563-4a32-bd95-4dd4685bcbe7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:20.704397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48c244-b563-4a32-bd95-4dd4685bcbe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:20.705754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48c244-b563-4a32-bd95-4dd4685bcbe7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:20.713228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 569/1166] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Aug 26 15:33:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:22.558336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f65af9-17b2-4193-ae1c-7ce0f8ef73be None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:22.558900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f65af9-17b2-4193-ae1c-7ce0f8ef73be None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:22.558900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f65af9-17b2-4193-ae1c-7ce0f8ef73be None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:22.560252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 598/1167] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:22.565285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1417186-7bdb-4fc3-aaa8-681f5aafa7b2 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:22.565729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1417186-7bdb-4fc3-aaa8-681f5aafa7b2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:22.565839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1417186-7bdb-4fc3-aaa8-681f5aafa7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:22.583501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b1417186-7bdb-4fc3-aaa8-681f5aafa7b2 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:22.584729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b1417186-7bdb-4fc3-aaa8-681f5aafa7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:22.610793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:22.649932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 570/1168] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:22.919557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f65af9-17b2-4193-ae1c-7ce0f8ef73be None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:23.023116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f65af9-17b2-4193-ae1c-7ce0f8ef73be None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:23.036615 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:23.056425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322400a-39c4-4ede-b4ae-2d7c2a113fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:23.063382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322400a-39c4-4ede-b4ae-2d7c2a113fb7 None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:23.063382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322400a-39c4-4ede-b4ae-2d7c2a113fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:23.063382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322400a-39c4-4ede-b4ae-2d7c2a113fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:23.063382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322400a-39c4-4ede-b4ae-2d7c2a113fb7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:23.063382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322400a-39c4-4ede-b4ae-2d7c2a113fb7 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:23.143898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 599/1169] 10.222.0.98 () {60 vars in 1138 bytes} [Fri Aug 26 15:33:22 2022] POST /identity/v3/domains => generated 267 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:33:25.081920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfa795-14fa-4edc-bd23-e8bc0ba49c84 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:25.082467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfa795-14fa-4edc-bd23-e8bc0ba49c84 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:25.082467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfa795-14fa-4edc-bd23-e8bc0ba49c84 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:25.083797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 571/1170] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:25.089666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299809-d308-4c4b-a885-30f4db1a1e6e None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:25.090113 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299809-d308-4c4b-a885-30f4db1a1e6e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:25.090237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299809-d308-4c4b-a885-30f4db1a1e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:25.107814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-75299809-d308-4c4b-a885-30f4db1a1e6e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:25.109037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-75299809-d308-4c4b-a885-30f4db1a1e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:25.174332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 600/1171] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:25.440005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfa795-14fa-4edc-bd23-e8bc0ba49c84 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:25.537585 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfa795-14fa-4edc-bd23-e8bc0ba49c84 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:25.570263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a71bc5-6ac9-40bc-ab5c-ccf7f62d4440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:25.570871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a71bc5-6ac9-40bc-ab5c-ccf7f62d4440 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:25.571150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a71bc5-6ac9-40bc-ab5c-ccf7f62d4440 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:25.571422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a71bc5-6ac9-40bc-ab5c-ccf7f62d4440 None admin] PATH_INFO: `/v3/domains/da545700884c4f80bcd51c4bd278098a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:25.581939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a71bc5-6ac9-40bc-ab5c-ccf7f62d4440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=da545700884c4f80bcd51c4bd278098a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:25.583456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a71bc5-6ac9-40bc-ab5c-ccf7f62d4440 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:25.588136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 572/1172] 10.222.0.98 () {56 vars in 1183 bytes} [Fri Aug 26 15:33:25 2022] GET /identity/v3/domains/da545700884c4f80bcd51c4bd278098a => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:25.593946 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75299809-d308-4c4b-a885-30f4db1a1e6e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:25.689153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75299809-d308-4c4b-a885-30f4db1a1e6e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:25.690693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:25.724853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d091a-d560-4f61-b964-b5a3666b1971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:25.725465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d091a-d560-4f61-b964-b5a3666b1971 None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:25.725969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d091a-d560-4f61-b964-b5a3666b1971 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:25.726622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d091a-d560-4f61-b964-b5a3666b1971 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:25.727960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d091a-d560-4f61-b964-b5a3666b1971 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:25.729404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d091a-d560-4f61-b964-b5a3666b1971 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:25.824352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 601/1173] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:33:25 2022] POST /identity/v3/projects => generated 361 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:33:27.808850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eb0bf-af08-4be4-ba33-063afefffd32 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:27.809390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eb0bf-af08-4be4-ba33-063afefffd32 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:27.809390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eb0bf-af08-4be4-ba33-063afefffd32 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:27.810738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 573/1174] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:27.815958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eed90-db45-46d5-974c-28c3af1e7611 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:27.816336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eed90-db45-46d5-974c-28c3af1e7611 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:27.816515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eed90-db45-46d5-974c-28c3af1e7611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:27.834749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-bd0eed90-db45-46d5-974c-28c3af1e7611 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:27.835999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bd0eed90-db45-46d5-974c-28c3af1e7611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:27.901986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 602/1175] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:28.167691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0eb0bf-af08-4be4-ba33-063afefffd32 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:28.192318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:28.269785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0eb0bf-af08-4be4-ba33-063afefffd32 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:28.302865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aea393-e15d-4c1f-8f0f-467e062c6e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:28.303483 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aea393-e15d-4c1f-8f0f-467e062c6e15 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:28.303769 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aea393-e15d-4c1f-8f0f-467e062c6e15 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:28.304075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aea393-e15d-4c1f-8f0f-467e062c6e15 None admin] PATH_INFO: `/v3/domains/da545700884c4f80bcd51c4bd278098a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:28.305217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aea393-e15d-4c1f-8f0f-467e062c6e15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=da545700884c4f80bcd51c4bd278098a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:28.306646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aea393-e15d-4c1f-8f0f-467e062c6e15 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:28.308230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 574/1176] 10.222.0.98 () {56 vars in 1183 bytes} [Fri Aug 26 15:33:28 2022] GET /identity/v3/domains/da545700884c4f80bcd51c4bd278098a => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:28.314169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0eed90-db45-46d5-974c-28c3af1e7611 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:28.395124 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:28.409896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0eed90-db45-46d5-974c-28c3af1e7611 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:28.443137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1fe7c-37fa-4ee8-9f6b-901aa6c3ed2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:28.443718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1fe7c-37fa-4ee8-9f6b-901aa6c3ed2c None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:28.444035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1fe7c-37fa-4ee8-9f6b-901aa6c3ed2c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:28.444322 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1fe7c-37fa-4ee8-9f6b-901aa6c3ed2c None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:28.444991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1fe7c-37fa-4ee8-9f6b-901aa6c3ed2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:28.446933 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1fe7c-37fa-4ee8-9f6b-901aa6c3ed2c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:28.560216 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 603/1177] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:33:28 2022] POST /identity/v3/users => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:33:30.489126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd9fdc-c1a9-4253-8f53-79bc0a6cb1ce None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:30.489695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd9fdc-c1a9-4253-8f53-79bc0a6cb1ce None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:30.489695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd9fdc-c1a9-4253-8f53-79bc0a6cb1ce None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:30.491022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 575/1178] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:30.496498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66628d-0438-480e-9bf1-ebe213b5e943 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:30.496879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66628d-0438-480e-9bf1-ebe213b5e943 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:30.497058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66628d-0438-480e-9bf1-ebe213b5e943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:30.515269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ff66628d-0438-480e-9bf1-ebe213b5e943 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:30.516487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ff66628d-0438-480e-9bf1-ebe213b5e943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:30.581007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 604/1179] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:30.869417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd9fdc-c1a9-4253-8f53-79bc0a6cb1ce None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:30.957168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:30.967592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd9fdc-c1a9-4253-8f53-79bc0a6cb1ce None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:30.999801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703e782-d855-4268-8206-dfca620ba5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:31.000465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e782-d855-4268-8206-dfca620ba5dd None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:31.000743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e782-d855-4268-8206-dfca620ba5dd None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:31.001013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e782-d855-4268-8206-dfca620ba5dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:31.011673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703e782-d855-4268-8206-dfca620ba5dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-d703e782-d855-4268-8206-dfca620ba5dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:33:31.016566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:33:31.017648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:33:31.018615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 576/1180] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:33:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:33:31.025833 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66628d-0438-480e-9bf1-ebe213b5e943 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:31.092554 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:31.121136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66628d-0438-480e-9bf1-ebe213b5e943 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:31.154793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac5c35-95a6-46e8-94c7-9cd54df89b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:31.155387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac5c35-95a6-46e8-94c7-9cd54df89b7e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:31.155675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac5c35-95a6-46e8-94c7-9cd54df89b7e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:31.155987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac5c35-95a6-46e8-94c7-9cd54df89b7e None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:31.156651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac5c35-95a6-46e8-94c7-9cd54df89b7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:31.156959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac5c35-95a6-46e8-94c7-9cd54df89b7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:31.158368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac5c35-95a6-46e8-94c7-9cd54df89b7e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:31.167217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 605/1181] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:33:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:31.173071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703e782-d855-4268-8206-dfca620ba5dd None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:31.270230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703e782-d855-4268-8206-dfca620ba5dd None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:31.303217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a7553-dd42-4e1e-ae77-93fd7acba0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:31.303834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a7553-dd42-4e1e-ae77-93fd7acba0df None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:31.304193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a7553-dd42-4e1e-ae77-93fd7acba0df None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:31.304475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a7553-dd42-4e1e-ae77-93fd7acba0df None admin] PATH_INFO: `/v3/users/f0270fb5fea44926aa7f45527aa8060b` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:31.316995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a7553-dd42-4e1e-ae77-93fd7acba0df None admin] RBAC: Authorizing `identity:get_user(user_id=f0270fb5fea44926aa7f45527aa8060b)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:31.318433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a7553-dd42-4e1e-ae77-93fd7acba0df None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:31.320546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 577/1182] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:33:31 2022] GET /identity/v3/users/f0270fb5fea44926aa7f45527aa8060b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:31.326434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac5c35-95a6-46e8-94c7-9cd54df89b7e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:31.422865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac5c35-95a6-46e8-94c7-9cd54df89b7e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:31.451739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:31.456672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741af1a-8060-404e-abb6-5693fae35f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:31.457264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741af1a-8060-404e-abb6-5693fae35f9a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:31.457549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741af1a-8060-404e-abb6-5693fae35f9a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:31.457826 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741af1a-8060-404e-abb6-5693fae35f9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:31.524566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741af1a-8060-404e-abb6-5693fae35f9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:31.526022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741af1a-8060-404e-abb6-5693fae35f9a None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:31.684940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 606/1183] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:33:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:33:31.690837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a7553-dd42-4e1e-ae77-93fd7acba0df None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:31.702145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:31.800325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a7553-dd42-4e1e-ae77-93fd7acba0df None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:31.843324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c587f-a739-4094-a727-b57ace8764d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:31.843978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c587f-a739-4094-a727-b57ace8764d5 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:31.844265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c587f-a739-4094-a727-b57ace8764d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:31.844542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c587f-a739-4094-a727-b57ace8764d5 None admin] PATH_INFO: `/v3/projects/f66db20953ea4bc5b6f890bedc63346d` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:31.845654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c587f-a739-4094-a727-b57ace8764d5 None admin] RBAC: Authorizing `identity:get_project(project_id=f66db20953ea4bc5b6f890bedc63346d)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:31.847083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c587f-a739-4094-a727-b57ace8764d5 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:31.848738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 578/1184] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:33:31 2022] GET /identity/v3/projects/f66db20953ea4bc5b6f890bedc63346d => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:31.854935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741af1a-8060-404e-abb6-5693fae35f9a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:31.884759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:31.951581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741af1a-8060-404e-abb6-5693fae35f9a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:31.984116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81cfd7-228e-43b2-917b-f7651ff03c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:31.984703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81cfd7-228e-43b2-917b-f7651ff03c59 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:31.984989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81cfd7-228e-43b2-917b-f7651ff03c59 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:31.985266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81cfd7-228e-43b2-917b-f7651ff03c59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:31.985980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81cfd7-228e-43b2-917b-f7651ff03c59 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=f66db20953ea4bc5b6f890bedc63346d, user.id=f0270fb5fea44926aa7f45527aa8060b) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:31.986292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81cfd7-228e-43b2-917b-f7651ff03c59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:31.987709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81cfd7-228e-43b2-917b-f7651ff03c59 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:31.992957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 607/1185] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:33:31 2022] GET /identity/v3/role_assignments?user.id=f0270fb5fea44926aa7f45527aa8060b&scope.project.id=f66db20953ea4bc5b6f890bedc63346d&role.id=6ba5a5add2174983be37258aad1ee404 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:33.958740 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a8652-c64b-496a-a340-629bc4bb0b90 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:33.959246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a8652-c64b-496a-a340-629bc4bb0b90 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:33.959581 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a8652-c64b-496a-a340-629bc4bb0b90 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:33.961868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 579/1186] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:33.967090 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d1ee5-47a7-441e-87f5-d611fef52e82 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:33.967527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d1ee5-47a7-441e-87f5-d611fef52e82 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:33.967656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d1ee5-47a7-441e-87f5-d611fef52e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:33.985867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-c95d1ee5-47a7-441e-87f5-d611fef52e82 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:33.987055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c95d1ee5-47a7-441e-87f5-d611fef52e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:34.051540 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 608/1187] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:34.321176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a8652-c64b-496a-a340-629bc4bb0b90 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:34.396509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:34.419946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a8652-c64b-496a-a340-629bc4bb0b90 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:34.452861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2648b4c-5397-4fbe-b6f4-12f93dd87c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:34.453468 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2648b4c-5397-4fbe-b6f4-12f93dd87c98 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:34.453759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2648b4c-5397-4fbe-b6f4-12f93dd87c98 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:34.454081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2648b4c-5397-4fbe-b6f4-12f93dd87c98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:34.464416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2648b4c-5397-4fbe-b6f4-12f93dd87c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-e2648b4c-5397-4fbe-b6f4-12f93dd87c98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:33:34.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:33:34.470134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:33:34.471225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 580/1188] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:33:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:33:34.476716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d1ee5-47a7-441e-87f5-d611fef52e82 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:34.502463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:34.571806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d1ee5-47a7-441e-87f5-d611fef52e82 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:34.607989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e543a-7684-4097-8c8a-86a291ef7b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:34.608603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e543a-7684-4097-8c8a-86a291ef7b47 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:34.608891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e543a-7684-4097-8c8a-86a291ef7b47 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:34.609171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e543a-7684-4097-8c8a-86a291ef7b47 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:34.609837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e543a-7684-4097-8c8a-86a291ef7b47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:34.610145 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e543a-7684-4097-8c8a-86a291ef7b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:34.611524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e543a-7684-4097-8c8a-86a291ef7b47 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:34.619496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 609/1189] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:33:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:34.625356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2648b4c-5397-4fbe-b6f4-12f93dd87c98 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:34.721519 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2648b4c-5397-4fbe-b6f4-12f93dd87c98 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:34.754918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed515b24-5332-4eca-91e5-c8bb95a95534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:34.755537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed515b24-5332-4eca-91e5-c8bb95a95534 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:34.755824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed515b24-5332-4eca-91e5-c8bb95a95534 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:34.756135 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed515b24-5332-4eca-91e5-c8bb95a95534 None admin] PATH_INFO: `/v3/users/f0270fb5fea44926aa7f45527aa8060b` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:34.757741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed515b24-5332-4eca-91e5-c8bb95a95534 None admin] RBAC: Authorizing `identity:get_user(user_id=f0270fb5fea44926aa7f45527aa8060b)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:34.759187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed515b24-5332-4eca-91e5-c8bb95a95534 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:34.760776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 581/1190] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:33:34 2022] GET /identity/v3/users/f0270fb5fea44926aa7f45527aa8060b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:34.766470 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e543a-7684-4097-8c8a-86a291ef7b47 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:34.852117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:34.863576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e543a-7684-4097-8c8a-86a291ef7b47 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:34.896353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd9e92-be95-47d5-86f8-2ad0675dc8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:34.896966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd9e92-be95-47d5-86f8-2ad0675dc8f6 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:34.897251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd9e92-be95-47d5-86f8-2ad0675dc8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:34.897531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd9e92-be95-47d5-86f8-2ad0675dc8f6 None admin] PATH_INFO: `/v3/projects/f66db20953ea4bc5b6f890bedc63346d` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:34.899224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd9e92-be95-47d5-86f8-2ad0675dc8f6 None admin] RBAC: Authorizing `identity:get_project(project_id=f66db20953ea4bc5b6f890bedc63346d)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:34.900676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd9e92-be95-47d5-86f8-2ad0675dc8f6 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:34.902268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 610/1191] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:33:34 2022] GET /identity/v3/projects/f66db20953ea4bc5b6f890bedc63346d => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:34.908297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed515b24-5332-4eca-91e5-c8bb95a95534 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:34.912035 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:35.005978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed515b24-5332-4eca-91e5-c8bb95a95534 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:35.038580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7f536-6a63-4a21-9d1c-148c596f3385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:35.039218 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f536-6a63-4a21-9d1c-148c596f3385 None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:35.039496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f536-6a63-4a21-9d1c-148c596f3385 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:35.039765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f536-6a63-4a21-9d1c-148c596f3385 None admin] PATH_INFO: `/v3/projects/f66db20953ea4bc5b6f890bedc63346d/users/f0270fb5fea44926aa7f45527aa8060b/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:35.041895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7f536-6a63-4a21-9d1c-148c596f3385 None admin] RBAC: Authorizing `identity:create_grant(project_id=f66db20953ea4bc5b6f890bedc63346d, user_id=f0270fb5fea44926aa7f45527aa8060b, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:35.043308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7f536-6a63-4a21-9d1c-148c596f3385 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:35.112978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 582/1192] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:33:34 2022] PUT /identity/v3/projects/f66db20953ea4bc5b6f890bedc63346d/users/f0270fb5fea44926aa7f45527aa8060b/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:33:36.971751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db484f8f-2e58-49a7-88b1-4c9a9e754f9c None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:36.972409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db484f8f-2e58-49a7-88b1-4c9a9e754f9c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:36.972409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db484f8f-2e58-49a7-88b1-4c9a9e754f9c None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:36.973706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 611/1193] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:33:36.979457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af004971-d7df-485d-ab03-3e3363706125 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:36.979892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af004971-d7df-485d-ab03-3e3363706125 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:36.980048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af004971-d7df-485d-ab03-3e3363706125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:36.997654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-af004971-d7df-485d-ab03-3e3363706125 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:33:36.998831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-af004971-d7df-485d-ab03-3e3363706125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:33:37.058091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:37.066867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 583/1194] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:33:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:33:37.338968 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db484f8f-2e58-49a7-88b1-4c9a9e754f9c None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:37.437560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db484f8f-2e58-49a7-88b1-4c9a9e754f9c None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:37.470136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb86d41-8ba7-46de-a354-d51fab6c253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:37.470743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb86d41-8ba7-46de-a354-d51fab6c253a None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:37.471020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb86d41-8ba7-46de-a354-d51fab6c253a None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:37.471290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb86d41-8ba7-46de-a354-d51fab6c253a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:37.481078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb86d41-8ba7-46de-a354-d51fab6c253a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-eeb86d41-8ba7-46de-a354-d51fab6c253a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:33:37.485770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:33:37.487008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:33:37.488296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 612/1195] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:33:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:33:37.493485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af004971-d7df-485d-ab03-3e3363706125 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:37.588524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af004971-d7df-485d-ab03-3e3363706125 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:37.621350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861fab5-78e3-45d7-b300-099701d1e0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:37.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861fab5-78e3-45d7-b300-099701d1e0ba None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:37.622252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861fab5-78e3-45d7-b300-099701d1e0ba None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:37.625003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861fab5-78e3-45d7-b300-099701d1e0ba None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:37.625003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861fab5-78e3-45d7-b300-099701d1e0ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:37.625003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861fab5-78e3-45d7-b300-099701d1e0ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:37.625003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861fab5-78e3-45d7-b300-099701d1e0ba None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:37.633142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 584/1196] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:33:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:37.638968 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb86d41-8ba7-46de-a354-d51fab6c253a None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:37.657126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:37.735622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb86d41-8ba7-46de-a354-d51fab6c253a None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:37.768151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9749103-ec87-4716-8589-0cda9cec0af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:37.768759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9749103-ec87-4716-8589-0cda9cec0af3 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:37.769043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9749103-ec87-4716-8589-0cda9cec0af3 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:37.769317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9749103-ec87-4716-8589-0cda9cec0af3 None admin] PATH_INFO: `/v3/users/f0270fb5fea44926aa7f45527aa8060b` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:37.770423 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9749103-ec87-4716-8589-0cda9cec0af3 None admin] RBAC: Authorizing `identity:get_user(user_id=f0270fb5fea44926aa7f45527aa8060b)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:37.771839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9749103-ec87-4716-8589-0cda9cec0af3 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:37.774034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 613/1197] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Aug 26 15:33:37 2022] GET /identity/v3/users/f0270fb5fea44926aa7f45527aa8060b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:33:37.780062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861fab5-78e3-45d7-b300-099701d1e0ba None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:37.834709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:37.876416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861fab5-78e3-45d7-b300-099701d1e0ba None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:37.908950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898735b-7c0e-4811-a50c-0634f9b487ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:37.909542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898735b-7c0e-4811-a50c-0634f9b487ca None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:37.909821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898735b-7c0e-4811-a50c-0634f9b487ca None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:37.910094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898735b-7c0e-4811-a50c-0634f9b487ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:37.974814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898735b-7c0e-4811-a50c-0634f9b487ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:37.976301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898735b-7c0e-4811-a50c-0634f9b487ca None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:38.027575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:38.069061 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 585/1198] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:33:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:33:38.075529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9749103-ec87-4716-8589-0cda9cec0af3 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:38.157765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:38.173274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9749103-ec87-4716-8589-0cda9cec0af3 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:38.207147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf4a11-cb1b-4e12-b322-3bf1a30a15f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:38.207769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf4a11-cb1b-4e12-b322-3bf1a30a15f3 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:38.208094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf4a11-cb1b-4e12-b322-3bf1a30a15f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:38.208376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf4a11-cb1b-4e12-b322-3bf1a30a15f3 None admin] PATH_INFO: `/v3/projects/f66db20953ea4bc5b6f890bedc63346d` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:38.209489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf4a11-cb1b-4e12-b322-3bf1a30a15f3 None admin] RBAC: Authorizing `identity:get_project(project_id=f66db20953ea4bc5b6f890bedc63346d)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:38.210917 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf4a11-cb1b-4e12-b322-3bf1a30a15f3 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:38.212564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 614/1199] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Aug 26 15:33:38 2022] GET /identity/v3/projects/f66db20953ea4bc5b6f890bedc63346d => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:38.218688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898735b-7c0e-4811-a50c-0634f9b487ca None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:33:38.315084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898735b-7c0e-4811-a50c-0634f9b487ca None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:33:38.348915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a013d0d-3924-4059-b2d9-97aa01f994a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:33:38.349515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a013d0d-3924-4059-b2d9-97aa01f994a6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:33:38.349797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a013d0d-3924-4059-b2d9-97aa01f994a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:33:38.350071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a013d0d-3924-4059-b2d9-97aa01f994a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:33:38.350820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a013d0d-3924-4059-b2d9-97aa01f994a6 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=f66db20953ea4bc5b6f890bedc63346d, user.id=f0270fb5fea44926aa7f45527aa8060b) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:33:38.351131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a013d0d-3924-4059-b2d9-97aa01f994a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:33:38.352592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a013d0d-3924-4059-b2d9-97aa01f994a6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:33:38.358189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 586/1200] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:33:38 2022] GET /identity/v3/role_assignments?user.id=f0270fb5fea44926aa7f45527aa8060b&scope.project.id=f66db20953ea4bc5b6f890bedc63346d&role.id=6ba5a5add2174983be37258aad1ee404 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:33:48.202989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=24 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:33:48.355613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:54.136960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28480bba-3311-4a55-ba16-cfa1df10001e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:54.137546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28480bba-3311-4a55-ba16-cfa1df10001e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:54.137651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28480bba-3311-4a55-ba16-cfa1df10001e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:54.139067 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 615/1201] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:34:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:54.144354 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f085ee-ba9b-4678-b2cd-330665b8ca39 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:54.144833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f085ee-ba9b-4678-b2cd-330665b8ca39 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:54.144961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f085ee-ba9b-4678-b2cd-330665b8ca39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:54.163731 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-46f085ee-ba9b-4678-b2cd-330665b8ca39 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:54.165026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-46f085ee-ba9b-4678-b2cd-330665b8ca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:54.233469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 587/1202] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:34:54.501981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28480bba-3311-4a55-ba16-cfa1df10001e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:54.605679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28480bba-3311-4a55-ba16-cfa1df10001e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:54.640363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c66899-f294-4a24-9cb8-b540b6fb6370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:54.645114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c66899-f294-4a24-9cb8-b540b6fb6370 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:54.645114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c66899-f294-4a24-9cb8-b540b6fb6370 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:54.645114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c66899-f294-4a24-9cb8-b540b6fb6370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:54.651487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c66899-f294-4a24-9cb8-b540b6fb6370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:54.652961 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c66899-f294-4a24-9cb8-b540b6fb6370 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-33c66899-f294-4a24-9cb8-b540b6fb6370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:34:54.667370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:54.668691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:34:54.669675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:34:54.669675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:34:54.669675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:34:54.669675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 616/1203] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:54.675106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f085ee-ba9b-4678-b2cd-330665b8ca39 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:54.773651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f085ee-ba9b-4678-b2cd-330665b8ca39 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:54.801425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:54.807856 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bbd13-eb99-44ee-b1a5-5590ec949db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:54.808500 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bbd13-eb99-44ee-b1a5-5590ec949db7 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:54.808785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bbd13-eb99-44ee-b1a5-5590ec949db7 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:54.809060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bbd13-eb99-44ee-b1a5-5590ec949db7 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:54.809705 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bbd13-eb99-44ee-b1a5-5590ec949db7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:54.810009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bbd13-eb99-44ee-b1a5-5590ec949db7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:54.811370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bbd13-eb99-44ee-b1a5-5590ec949db7 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:54.823060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 588/1204] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:54.830311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c66899-f294-4a24-9cb8-b540b6fb6370 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:54.882739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:54.930005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c66899-f294-4a24-9cb8-b540b6fb6370 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:54.963751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa3fac-68a9-4e58-8f07-5297cfad70fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:54.964427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa3fac-68a9-4e58-8f07-5297cfad70fe None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:54.964759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa3fac-68a9-4e58-8f07-5297cfad70fe None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:54.965073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa3fac-68a9-4e58-8f07-5297cfad70fe None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:54.965756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa3fac-68a9-4e58-8f07-5297cfad70fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:54.967335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa3fac-68a9-4e58-8f07-5297cfad70fe None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:55.077148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 617/1205] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Aug 26 15:34:54 2022] POST /identity/v3/users => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:34:57.021322 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac413d0-0bf3-4556-9532-004ea98c27de None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.022373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac413d0-0bf3-4556-9532-004ea98c27de None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.022373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac413d0-0bf3-4556-9532-004ea98c27de None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.023708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 589/1206] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:57.029239 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb8594-f312-4566-afc1-1d96878625a8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.029692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb8594-f312-4566-afc1-1d96878625a8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.029966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb8594-f312-4566-afc1-1d96878625a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.049106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-b0bb8594-f312-4566-afc1-1d96878625a8 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:57.050428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-b0bb8594-f312-4566-afc1-1d96878625a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:57.117736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 618/1207] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:34:57.398854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac413d0-0bf3-4556-9532-004ea98c27de None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:57.504909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac413d0-0bf3-4556-9532-004ea98c27de None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:57.539649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cb4ca-42dc-4027-9e9a-54c8e94bab25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:57.540307 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cb4ca-42dc-4027-9e9a-54c8e94bab25 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.540598 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cb4ca-42dc-4027-9e9a-54c8e94bab25 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.541137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cb4ca-42dc-4027-9e9a-54c8e94bab25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.551599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5cb4ca-42dc-4027-9e9a-54c8e94bab25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-6f5cb4ca-42dc-4027-9e9a-54c8e94bab25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:57.556790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:57.557946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:34:57.558822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 590/1208] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:34:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:57.564436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb8594-f312-4566-afc1-1d96878625a8 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:57.605305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:57.663339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb8594-f312-4566-afc1-1d96878625a8 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:57.699259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312481a6-4db1-4c1f-bf74-e86b49fb0e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:57.699891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312481a6-4db1-4c1f-bf74-e86b49fb0e36 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.700174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312481a6-4db1-4c1f-bf74-e86b49fb0e36 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.700449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312481a6-4db1-4c1f-bf74-e86b49fb0e36 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.701112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312481a6-4db1-4c1f-bf74-e86b49fb0e36 None admin] RBAC: Adding query filter params (name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:57.701413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312481a6-4db1-4c1f-bf74-e86b49fb0e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:57.702776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312481a6-4db1-4c1f-bf74-e86b49fb0e36 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:57.710932 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 619/1209] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:34:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:57.716804 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cb4ca-42dc-4027-9e9a-54c8e94bab25 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:57.768826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:57.817644 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cb4ca-42dc-4027-9e9a-54c8e94bab25 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:57.852205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510eb6f-55e4-4f29-b25e-0786ede6b88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:57.852832 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510eb6f-55e4-4f29-b25e-0786ede6b88c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.853110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510eb6f-55e4-4f29-b25e-0786ede6b88c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.853380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510eb6f-55e4-4f29-b25e-0786ede6b88c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.864950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510eb6f-55e4-4f29-b25e-0786ede6b88c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:57.867661 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510eb6f-55e4-4f29-b25e-0786ede6b88c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-d510eb6f-55e4-4f29-b25e-0786ede6b88c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:34:57.882239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:57.883968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:34:57.885230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:34:57.885230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:34:57.885230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:34:57.885230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 591/1210] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:34:57.889372 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312481a6-4db1-4c1f-bf74-e86b49fb0e36 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:57.987733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:57.991355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312481a6-4db1-4c1f-bf74-e86b49fb0e36 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.024910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68204cfb-6fcd-4e41-9b37-cd2aa2ad83ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.025512 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68204cfb-6fcd-4e41-9b37-cd2aa2ad83ec None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.025799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68204cfb-6fcd-4e41-9b37-cd2aa2ad83ec None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.026079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68204cfb-6fcd-4e41-9b37-cd2aa2ad83ec None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.026720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68204cfb-6fcd-4e41-9b37-cd2aa2ad83ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:58.027017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68204cfb-6fcd-4e41-9b37-cd2aa2ad83ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.028478 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68204cfb-6fcd-4e41-9b37-cd2aa2ad83ec None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.040458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 620/1211] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:58.046745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510eb6f-55e4-4f29-b25e-0786ede6b88c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.151463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510eb6f-55e4-4f29-b25e-0786ede6b88c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.153136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:58.188547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26b418-8be5-4715-b337-d016315878a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.189237 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26b418-8be5-4715-b337-d016315878a8 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.189629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26b418-8be5-4715-b337-d016315878a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.189982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26b418-8be5-4715-b337-d016315878a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.197428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26b418-8be5-4715-b337-d016315878a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.206897 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26b418-8be5-4715-b337-d016315878a8 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-ee26b418-8be5-4715-b337-d016315878a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:34:58.210825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:58.212039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:58.213197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:58.213197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:58.213197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:34:58.213197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:34:58.213197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:34:58.213197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:34:58.213197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:34:58.213197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:34:58.213197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:34:58.213197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:34:58.213197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 592/1212] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:58.217838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68204cfb-6fcd-4e41-9b37-cd2aa2ad83ec None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.319762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68204cfb-6fcd-4e41-9b37-cd2aa2ad83ec None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.354161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623a244-a63e-4bd3-8264-dd0aa5b23aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.354764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623a244-a63e-4bd3-8264-dd0aa5b23aca None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.355045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623a244-a63e-4bd3-8264-dd0aa5b23aca None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.355317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623a244-a63e-4bd3-8264-dd0aa5b23aca None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.356079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623a244-a63e-4bd3-8264-dd0aa5b23aca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:58.356382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623a244-a63e-4bd3-8264-dd0aa5b23aca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.357887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623a244-a63e-4bd3-8264-dd0aa5b23aca None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.375547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 621/1213] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Aug 26 15:34:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:58.381884 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26b418-8be5-4715-b337-d016315878a8 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.484356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26b418-8be5-4715-b337-d016315878a8 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.519295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aa798-d9cd-49ee-9213-e7e10e4a4785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.519925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aa798-d9cd-49ee-9213-e7e10e4a4785 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.520207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aa798-d9cd-49ee-9213-e7e10e4a4785 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.520480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aa798-d9cd-49ee-9213-e7e10e4a4785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.549660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:58.587726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aa798-d9cd-49ee-9213-e7e10e4a4785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.589119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aa798-d9cd-49ee-9213-e7e10e4a4785 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.689661 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 593/1214] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:34:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:34:58.695891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623a244-a63e-4bd3-8264-dd0aa5b23aca None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.715434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:58.800962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623a244-a63e-4bd3-8264-dd0aa5b23aca None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.835927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf944df-d223-45a2-ab85-56183c63c473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.836550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf944df-d223-45a2-ab85-56183c63c473 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.836828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf944df-d223-45a2-ab85-56183c63c473 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.837097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf944df-d223-45a2-ab85-56183c63c473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.847418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf944df-d223-45a2-ab85-56183c63c473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.849035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf944df-d223-45a2-ab85-56183c63c473 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-abf944df-d223-45a2-ab85-56183c63c473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:34:58.864284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:58.866803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:34:58.868823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:34:58.868823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:34:58.868823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:34:58.868823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 622/1215] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:58.873161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aa798-d9cd-49ee-9213-e7e10e4a4785 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.934316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:58.975025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aa798-d9cd-49ee-9213-e7e10e4a4785 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:59.010375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5a2c0-5060-4b9f-8733-fe0fde8f8404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:59.010982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5a2c0-5060-4b9f-8733-fe0fde8f8404 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:59.011262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5a2c0-5060-4b9f-8733-fe0fde8f8404 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:59.011535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5a2c0-5060-4b9f-8733-fe0fde8f8404 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:59.012293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5a2c0-5060-4b9f-8733-fe0fde8f8404 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:59.012597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5a2c0-5060-4b9f-8733-fe0fde8f8404 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:59.014064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5a2c0-5060-4b9f-8733-fe0fde8f8404 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:59.026501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 594/1216] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:59.032550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf944df-d223-45a2-ab85-56183c63c473 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:59.106465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:59.137676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf944df-d223-45a2-ab85-56183c63c473 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:59.172808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f80b3-ff52-4991-9bdd-a13072500433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:59.173551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f80b3-ff52-4991-9bdd-a13072500433 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:59.173920 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f80b3-ff52-4991-9bdd-a13072500433 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:59.174296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f80b3-ff52-4991-9bdd-a13072500433 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:59.182442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f80b3-ff52-4991-9bdd-a13072500433 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:59.184191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f80b3-ff52-4991-9bdd-a13072500433 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-7a2f80b3-ff52-4991-9bdd-a13072500433 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:34:59.195764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:59.197041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:59.198191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:34:59.198191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:34:59.198191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:34:59.198191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:34:59.198191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:34:59.198191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:34:59.198191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:34:59.198191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:34:59.198191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 623/1217] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:34:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:59.202943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5a2c0-5060-4b9f-8733-fe0fde8f8404 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:59.305289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5a2c0-5060-4b9f-8733-fe0fde8f8404 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:59.328881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:59.340912 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098ebf7-99da-49e1-b025-05c47fcb7bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:59.341592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098ebf7-99da-49e1-b025-05c47fcb7bc8 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:59.341802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098ebf7-99da-49e1-b025-05c47fcb7bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:59.342078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098ebf7-99da-49e1-b025-05c47fcb7bc8 None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:59.342771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098ebf7-99da-49e1-b025-05c47fcb7bc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:59.343078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098ebf7-99da-49e1-b025-05c47fcb7bc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:59.344525 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098ebf7-99da-49e1-b025-05c47fcb7bc8 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:59.356336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 595/1218] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:34:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:59.362532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f80b3-ff52-4991-9bdd-a13072500433 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:59.463709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f80b3-ff52-4991-9bdd-a13072500433 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:59.491659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:59.497168 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45dea75-e068-4174-a735-a30b5886eeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:59.497770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dea75-e068-4174-a735-a30b5886eeb4 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:59.498057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dea75-e068-4174-a735-a30b5886eeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:59.498334 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dea75-e068-4174-a735-a30b5886eeb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:59.499051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45dea75-e068-4174-a735-a30b5886eeb4 None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=b28f987c90df4c19a12bb1819dc65cf9) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:59.499362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45dea75-e068-4174-a735-a30b5886eeb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:59.500819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45dea75-e068-4174-a735-a30b5886eeb4 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:59.506582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 624/1219] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:34:59 2022] GET /identity/v3/role_assignments?user.id=b28f987c90df4c19a12bb1819dc65cf9&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:01.419714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d3e33-6524-40e7-8c15-66b4ef5ccb14 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:01.420345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d3e33-6524-40e7-8c15-66b4ef5ccb14 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:01.420345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d3e33-6524-40e7-8c15-66b4ef5ccb14 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:01.421649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 596/1220] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:01.426790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1915a-704e-448d-a5c2-4b4aa87110ba None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:01.427254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1915a-704e-448d-a5c2-4b4aa87110ba None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:01.427388 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1915a-704e-448d-a5c2-4b4aa87110ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:01.446364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-c4f1915a-704e-448d-a5c2-4b4aa87110ba None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:01.447565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c4f1915a-704e-448d-a5c2-4b4aa87110ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:01.515071 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 625/1221] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:35:01.785817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d3e33-6524-40e7-8c15-66b4ef5ccb14 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:01.889214 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d3e33-6524-40e7-8c15-66b4ef5ccb14 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:01.923997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079af27-3154-4896-978a-6e4455e5cc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:01.924635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079af27-3154-4896-978a-6e4455e5cc0c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:01.924918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079af27-3154-4896-978a-6e4455e5cc0c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:01.925207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079af27-3154-4896-978a-6e4455e5cc0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:01.937848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079af27-3154-4896-978a-6e4455e5cc0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-f079af27-3154-4896-978a-6e4455e5cc0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:01.942534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:01.948126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:01.949771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 597/1222] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:01.951073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1915a-704e-448d-a5c2-4b4aa87110ba None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:02.050378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1915a-704e-448d-a5c2-4b4aa87110ba None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:02.074689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:02.085449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c74829-297f-444e-bef0-be5f7eb68bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:02.088362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c74829-297f-444e-bef0-be5f7eb68bfb None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:02.088362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c74829-297f-444e-bef0-be5f7eb68bfb None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:02.088362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c74829-297f-444e-bef0-be5f7eb68bfb None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:02.088362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c74829-297f-444e-bef0-be5f7eb68bfb None admin] RBAC: Adding query filter params (name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:02.088362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c74829-297f-444e-bef0-be5f7eb68bfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:02.088966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c74829-297f-444e-bef0-be5f7eb68bfb None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:02.097155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 626/1223] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:35:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:02.103091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079af27-3154-4896-978a-6e4455e5cc0c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:02.147678 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:02.204344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079af27-3154-4896-978a-6e4455e5cc0c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:02.239686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e961c-ba17-4799-a1d4-8563765e00c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:02.240865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e961c-ba17-4799-a1d4-8563765e00c0 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:02.241149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e961c-ba17-4799-a1d4-8563765e00c0 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:02.241428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e961c-ba17-4799-a1d4-8563765e00c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:02.251309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e961c-ba17-4799-a1d4-8563765e00c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:02.252848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e961c-ba17-4799-a1d4-8563765e00c0 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-8b3e961c-ba17-4799-a1d4-8563765e00c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:02.267791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:02.271215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:02.272413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:02.272413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:02.272413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:02.272413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 598/1224] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:02.275896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c74829-297f-444e-bef0-be5f7eb68bfb None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:02.382302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c74829-297f-444e-bef0-be5f7eb68bfb None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:02.424349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be4c7f-32fa-4095-9cea-0654f425e8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:02.425264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be4c7f-32fa-4095-9cea-0654f425e8b6 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:02.425548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be4c7f-32fa-4095-9cea-0654f425e8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:02.425827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be4c7f-32fa-4095-9cea-0654f425e8b6 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:02.426611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be4c7f-32fa-4095-9cea-0654f425e8b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:02.426795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be4c7f-32fa-4095-9cea-0654f425e8b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:02.428227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be4c7f-32fa-4095-9cea-0654f425e8b6 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:02.441500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 627/1225] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:02.449217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e961c-ba17-4799-a1d4-8563765e00c0 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:02.550942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:02.557754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e961c-ba17-4799-a1d4-8563765e00c0 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:02.593994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8253163-c166-4ced-a113-a46e35591d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:02.594762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8253163-c166-4ced-a113-a46e35591d59 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:02.595213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8253163-c166-4ced-a113-a46e35591d59 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:02.595588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8253163-c166-4ced-a113-a46e35591d59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:02.605198 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8253163-c166-4ced-a113-a46e35591d59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:02.606848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8253163-c166-4ced-a113-a46e35591d59 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-c8253163-c166-4ced-a113-a46e35591d59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:02.619761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:02.621973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:02.623829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:02.623829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:02.623829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:02.623829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:02.623829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:02.623829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:02.623829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:02.623829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:02.623829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:35:02.623829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:02.623829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 599/1226] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:35:02.627895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be4c7f-32fa-4095-9cea-0654f425e8b6 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:02.699255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:02.753556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be4c7f-32fa-4095-9cea-0654f425e8b6 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:02.786809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e35554-4df8-4a7c-bb1f-0bf17b95a34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:02.787960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e35554-4df8-4a7c-bb1f-0bf17b95a34e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:02.788241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e35554-4df8-4a7c-bb1f-0bf17b95a34e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:02.788601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e35554-4df8-4a7c-bb1f-0bf17b95a34e None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:02.789318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e35554-4df8-4a7c-bb1f-0bf17b95a34e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:02.789615 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e35554-4df8-4a7c-bb1f-0bf17b95a34e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:02.791277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e35554-4df8-4a7c-bb1f-0bf17b95a34e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:02.808965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 628/1227] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Aug 26 15:35:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:02.814669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8253163-c166-4ced-a113-a46e35591d59 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:02.913441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8253163-c166-4ced-a113-a46e35591d59 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:02.945873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec38988-9af1-4418-8535-55b0070015f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:02.946485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec38988-9af1-4418-8535-55b0070015f4 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:02.946763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec38988-9af1-4418-8535-55b0070015f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:02.947034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec38988-9af1-4418-8535-55b0070015f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:02.957301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec38988-9af1-4418-8535-55b0070015f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:02.958673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec38988-9af1-4418-8535-55b0070015f4 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-9ec38988-9af1-4418-8535-55b0070015f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:02.973432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:02.975126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:02.976397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:02.976397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:02.976397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:02.976397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 600/1228] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:02.981099 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e35554-4df8-4a7c-bb1f-0bf17b95a34e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:03.085074 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e35554-4df8-4a7c-bb1f-0bf17b95a34e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:03.087467 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:03.120611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36359d3-8270-4a95-8446-5ed77d276d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:03.127186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36359d3-8270-4a95-8446-5ed77d276d25 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.127186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36359d3-8270-4a95-8446-5ed77d276d25 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.127186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36359d3-8270-4a95-8446-5ed77d276d25 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.127186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36359d3-8270-4a95-8446-5ed77d276d25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:03.127186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36359d3-8270-4a95-8446-5ed77d276d25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:03.127186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36359d3-8270-4a95-8446-5ed77d276d25 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:03.135236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 629/1229] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:03.141316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec38988-9af1-4418-8535-55b0070015f4 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:03.151252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:03.240363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec38988-9af1-4418-8535-55b0070015f4 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:03.275168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7516dd-6653-4086-9ca8-9dc5af4a2b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:03.275973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7516dd-6653-4086-9ca8-9dc5af4a2b6f None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.276261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7516dd-6653-4086-9ca8-9dc5af4a2b6f None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.276540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7516dd-6653-4086-9ca8-9dc5af4a2b6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.283415 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7516dd-6653-4086-9ca8-9dc5af4a2b6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:03.285632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7516dd-6653-4086-9ca8-9dc5af4a2b6f None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-7c7516dd-6653-4086-9ca8-9dc5af4a2b6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:03.297326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:03.299048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:03.300866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:03.300866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:03.300866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:03.300866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:03.300866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:03.300866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:03.300866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:03.300866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:03.300866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 601/1230] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:03.304717 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36359d3-8270-4a95-8446-5ed77d276d25 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:03.417107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36359d3-8270-4a95-8446-5ed77d276d25 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:03.453899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee336c-b3e8-48cb-b193-b8558f4442ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:03.454627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee336c-b3e8-48cb-b193-b8558f4442ee None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.454808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee336c-b3e8-48cb-b193-b8558f4442ee None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.455086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee336c-b3e8-48cb-b193-b8558f4442ee None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.455782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee336c-b3e8-48cb-b193-b8558f4442ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:03.456122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee336c-b3e8-48cb-b193-b8558f4442ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:03.457678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee336c-b3e8-48cb-b193-b8558f4442ee None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:03.470772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 630/1231] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:03.477138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7516dd-6653-4086-9ca8-9dc5af4a2b6f None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:03.543714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:03.580290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7516dd-6653-4086-9ca8-9dc5af4a2b6f None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:03.614459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143a861-c7d8-4c13-a6ab-bed99f342363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:03.615638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143a861-c7d8-4c13-a6ab-bed99f342363 None admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.615960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143a861-c7d8-4c13-a6ab-bed99f342363 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.616242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143a861-c7d8-4c13-a6ab-bed99f342363 None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/b28f987c90df4c19a12bb1819dc65cf9/roles/9dc2d706af5146e0b8d62ef94c74906d` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.630415 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143a861-c7d8-4c13-a6ab-bed99f342363 None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=b28f987c90df4c19a12bb1819dc65cf9, role_id=9dc2d706af5146e0b8d62ef94c74906d)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:03.632516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143a861-c7d8-4c13-a6ab-bed99f342363 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:03.752435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 602/1232] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:35:03 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/b28f987c90df4c19a12bb1819dc65cf9/roles/9dc2d706af5146e0b8d62ef94c74906d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:35:05.661480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52ee03-9cc3-42c7-9c1a-97bb9152cb29 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.662291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52ee03-9cc3-42c7-9c1a-97bb9152cb29 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.662291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52ee03-9cc3-42c7-9c1a-97bb9152cb29 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.663391 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 631/1233] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:05.669535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292430fd-1ade-45b7-9e6f-3649b7392d26 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.672952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292430fd-1ade-45b7-9e6f-3649b7392d26 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.672952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292430fd-1ade-45b7-9e6f-3649b7392d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.688194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-292430fd-1ade-45b7-9e6f-3649b7392d26 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:05.689403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-292430fd-1ade-45b7-9e6f-3649b7392d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:05.757452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 603/1234] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:35:06.033914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52ee03-9cc3-42c7-9c1a-97bb9152cb29 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.060019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.136910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52ee03-9cc3-42c7-9c1a-97bb9152cb29 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.171684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8984f45-a8a3-4bc7-a478-ac7d61afa91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.172339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8984f45-a8a3-4bc7-a478-ac7d61afa91d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.172658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8984f45-a8a3-4bc7-a478-ac7d61afa91d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.172934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8984f45-a8a3-4bc7-a478-ac7d61afa91d None admin] PATH_INFO: `/v3/roles/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.182804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8984f45-a8a3-4bc7-a478-ac7d61afa91d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-e8984f45-a8a3-4bc7-a478-ac7d61afa91d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:06.188455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:06.190203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:06.191923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:06.195103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 632/1235] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:06.196416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292430fd-1ade-45b7-9e6f-3649b7392d26 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.242936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.298864 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292430fd-1ade-45b7-9e6f-3649b7392d26 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.333646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62e7db-3c78-477e-ace3-fb06adf09c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.334883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62e7db-3c78-477e-ace3-fb06adf09c2b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.335168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62e7db-3c78-477e-ace3-fb06adf09c2b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.335448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62e7db-3c78-477e-ace3-fb06adf09c2b None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.336158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62e7db-3c78-477e-ace3-fb06adf09c2b None admin] RBAC: Adding query filter params (name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:06.336463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62e7db-3c78-477e-ace3-fb06adf09c2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.337880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62e7db-3c78-477e-ace3-fb06adf09c2b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.351830 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 604/1236] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:06.352798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8984f45-a8a3-4bc7-a478-ac7d61afa91d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.438377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.454416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8984f45-a8a3-4bc7-a478-ac7d61afa91d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.489118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a4cd-c091-4618-aa9e-05ae398653e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.489749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a4cd-c091-4618-aa9e-05ae398653e6 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.490065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a4cd-c091-4618-aa9e-05ae398653e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.490346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a4cd-c091-4618-aa9e-05ae398653e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.500517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394a4cd-c091-4618-aa9e-05ae398653e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.501962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394a4cd-c091-4618-aa9e-05ae398653e6 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-1394a4cd-c091-4618-aa9e-05ae398653e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:06.517460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:06.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:06.519703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:06.519703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:06.519703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:06.519703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 633/1237] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:35:06.525242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62e7db-3c78-477e-ace3-fb06adf09c2b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.626682 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62e7db-3c78-477e-ace3-fb06adf09c2b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.636643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.661752 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da01c5-ac5b-47dc-a63f-0d0d2bfb5ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.662383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da01c5-ac5b-47dc-a63f-0d0d2bfb5ddb None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.662675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da01c5-ac5b-47dc-a63f-0d0d2bfb5ddb None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.662950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da01c5-ac5b-47dc-a63f-0d0d2bfb5ddb None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.663597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da01c5-ac5b-47dc-a63f-0d0d2bfb5ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:06.663931 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da01c5-ac5b-47dc-a63f-0d0d2bfb5ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.665297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da01c5-ac5b-47dc-a63f-0d0d2bfb5ddb None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.677040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 605/1238] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:06.683944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a4cd-c091-4618-aa9e-05ae398653e6 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.786774 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a4cd-c091-4618-aa9e-05ae398653e6 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.821637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada9e59-a6be-4224-92f6-af777e0a6927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.822264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada9e59-a6be-4224-92f6-af777e0a6927 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.822587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada9e59-a6be-4224-92f6-af777e0a6927 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.822879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada9e59-a6be-4224-92f6-af777e0a6927 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.825817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.830897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada9e59-a6be-4224-92f6-af777e0a6927 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.832383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada9e59-a6be-4224-92f6-af777e0a6927 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-3ada9e59-a6be-4224-92f6-af777e0a6927 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:06.843477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:06.844596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:06.845659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:06.845659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:06.845659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:06.845659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:06.845659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:06.845659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:06.845659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:06.845659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:06.845659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:35:06.845659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:06.845659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 634/1239] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:06.850637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da01c5-ac5b-47dc-a63f-0d0d2bfb5ddb None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.950297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da01c5-ac5b-47dc-a63f-0d0d2bfb5ddb None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.984903 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b9ca4-5d16-44d4-a6ec-4edf28583d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.985511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b9ca4-5d16-44d4-a6ec-4edf28583d1d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.985854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b9ca4-5d16-44d4-a6ec-4edf28583d1d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.986078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b9ca4-5d16-44d4-a6ec-4edf28583d1d None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.986804 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b9ca4-5d16-44d4-a6ec-4edf28583d1d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:06.987111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b9ca4-5d16-44d4-a6ec-4edf28583d1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.988630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b9ca4-5d16-44d4-a6ec-4edf28583d1d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:07.006458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 606/1240] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:07.012707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada9e59-a6be-4224-92f6-af777e0a6927 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:07.112956 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada9e59-a6be-4224-92f6-af777e0a6927 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:07.146121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904b048-f1d0-4f62-ba08-8b0b3839d85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:07.146726 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904b048-f1d0-4f62-ba08-8b0b3839d85e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:07.147012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904b048-f1d0-4f62-ba08-8b0b3839d85e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:07.147293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904b048-f1d0-4f62-ba08-8b0b3839d85e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:07.208486 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:07.214908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904b048-f1d0-4f62-ba08-8b0b3839d85e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:07.216409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904b048-f1d0-4f62-ba08-8b0b3839d85e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:07.315625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 635/1241] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:35:07.321668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b9ca4-5d16-44d4-a6ec-4edf28583d1d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:07.348864 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:07.435382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b9ca4-5d16-44d4-a6ec-4edf28583d1d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:07.470497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca24f40-748a-4ac1-8da2-a8e9b233233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:07.471053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca24f40-748a-4ac1-8da2-a8e9b233233d None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:07.471332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca24f40-748a-4ac1-8da2-a8e9b233233d None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:07.471607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca24f40-748a-4ac1-8da2-a8e9b233233d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:07.482191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca24f40-748a-4ac1-8da2-a8e9b233233d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:07.483658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca24f40-748a-4ac1-8da2-a8e9b233233d None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-eca24f40-748a-4ac1-8da2-a8e9b233233d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:07.498641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:07.499741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:07.500915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:07.500915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:07.500915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:07.500915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 607/1242] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:07.506421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904b048-f1d0-4f62-ba08-8b0b3839d85e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:07.601806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:07.608041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904b048-f1d0-4f62-ba08-8b0b3839d85e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:07.641108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89973d7-bd4c-4048-9024-a9f1eab428e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:07.641666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89973d7-bd4c-4048-9024-a9f1eab428e0 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:07.641951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89973d7-bd4c-4048-9024-a9f1eab428e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:07.642234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89973d7-bd4c-4048-9024-a9f1eab428e0 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:07.642876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89973d7-bd4c-4048-9024-a9f1eab428e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:07.643185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89973d7-bd4c-4048-9024-a9f1eab428e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:07.644602 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89973d7-bd4c-4048-9024-a9f1eab428e0 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:07.656306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 636/1243] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:07.663641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca24f40-748a-4ac1-8da2-a8e9b233233d None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:07.743099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:07.764665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca24f40-748a-4ac1-8da2-a8e9b233233d None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:07.798697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24eccb6-ab78-462b-b581-cda61b33ca82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:07.799318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eccb6-ab78-462b-b581-cda61b33ca82 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:07.799596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eccb6-ab78-462b-b581-cda61b33ca82 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:07.799890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eccb6-ab78-462b-b581-cda61b33ca82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:07.806852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24eccb6-ab78-462b-b581-cda61b33ca82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:07.816028 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24eccb6-ab78-462b-b581-cda61b33ca82 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-f24eccb6-ab78-462b-b581-cda61b33ca82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:07.820423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:07.821877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:07.823078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:07.823078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:07.823078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:07.823078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:07.823078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:07.823078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:07.823078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:07.823078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:07.823078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 608/1244] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:07.827554 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89973d7-bd4c-4048-9024-a9f1eab428e0 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:07.933009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89973d7-bd4c-4048-9024-a9f1eab428e0 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:07.967157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a51f7-7aff-461a-a20f-cc0edfc79bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:07.967761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a51f7-7aff-461a-a20f-cc0edfc79bdf None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:07.968073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a51f7-7aff-461a-a20f-cc0edfc79bdf None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:07.968349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a51f7-7aff-461a-a20f-cc0edfc79bdf None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:07.969038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a51f7-7aff-461a-a20f-cc0edfc79bdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:07.969345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a51f7-7aff-461a-a20f-cc0edfc79bdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:07.970760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a51f7-7aff-461a-a20f-cc0edfc79bdf None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:07.982531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 637/1245] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:07.988829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24eccb6-ab78-462b-b581-cda61b33ca82 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:08.089261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24eccb6-ab78-462b-b581-cda61b33ca82 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:08.122306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f10b1-9f8f-43f2-8b8c-373866bbbf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:08.134584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f10b1-9f8f-43f2-8b8c-373866bbbf10 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:08.134584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f10b1-9f8f-43f2-8b8c-373866bbbf10 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:08.134584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f10b1-9f8f-43f2-8b8c-373866bbbf10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:08.134584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f10b1-9f8f-43f2-8b8c-373866bbbf10 None admin] RBAC: Adding query filter params (role.id=9dc2d706af5146e0b8d62ef94c74906d, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=b28f987c90df4c19a12bb1819dc65cf9) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:08.134584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f10b1-9f8f-43f2-8b8c-373866bbbf10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:08.134584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f10b1-9f8f-43f2-8b8c-373866bbbf10 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:08.135111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 609/1246] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:07 2022] GET /identity/v3/role_assignments?user.id=b28f987c90df4c19a12bb1819dc65cf9&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=9dc2d706af5146e0b8d62ef94c74906d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:10.103011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e375-91d3-41d2-8c34-a788a8dc0000 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.103911 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e375-91d3-41d2-8c34-a788a8dc0000 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.104070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e375-91d3-41d2-8c34-a788a8dc0000 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.105425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 638/1247] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:35:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:10.110910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019519e-4944-4286-8e7e-8c2580d0fbf5 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.111370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019519e-4944-4286-8e7e-8c2580d0fbf5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.111490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019519e-4944-4286-8e7e-8c2580d0fbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.118381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:10.130739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-4019519e-4944-4286-8e7e-8c2580d0fbf5 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:10.132009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4019519e-4944-4286-8e7e-8c2580d0fbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:10.198727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 610/1248] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:35:10.472497 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e375-91d3-41d2-8c34-a788a8dc0000 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.492498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:10.575098 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e375-91d3-41d2-8c34-a788a8dc0000 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.614685 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed6379-adf0-4fd1-9d03-9f3412322a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:10.615315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed6379-adf0-4fd1-9d03-9f3412322a05 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.615616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed6379-adf0-4fd1-9d03-9f3412322a05 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.615895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed6379-adf0-4fd1-9d03-9f3412322a05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.626088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed6379-adf0-4fd1-9d03-9f3412322a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-3eed6379-adf0-4fd1-9d03-9f3412322a05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:10.630839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:10.632048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:10.633003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 639/1249] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:10.638768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019519e-4944-4286-8e7e-8c2580d0fbf5 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.739643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019519e-4944-4286-8e7e-8c2580d0fbf5 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.768606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:10.773989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae0ed3-3e6f-4ea6-bf11-9e55f0ef0352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:10.774591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae0ed3-3e6f-4ea6-bf11-9e55f0ef0352 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.774880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae0ed3-3e6f-4ea6-bf11-9e55f0ef0352 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.775146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae0ed3-3e6f-4ea6-bf11-9e55f0ef0352 None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.775802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae0ed3-3e6f-4ea6-bf11-9e55f0ef0352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:10.776134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae0ed3-3e6f-4ea6-bf11-9e55f0ef0352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:10.777507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae0ed3-3e6f-4ea6-bf11-9e55f0ef0352 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:10.785706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 611/1250] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:10.791994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed6379-adf0-4fd1-9d03-9f3412322a05 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.870368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:10.893021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed6379-adf0-4fd1-9d03-9f3412322a05 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.926716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcd6d6-1e59-4718-bf81-6330284406df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:10.927362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcd6d6-1e59-4718-bf81-6330284406df None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.927653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcd6d6-1e59-4718-bf81-6330284406df None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.927972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcd6d6-1e59-4718-bf81-6330284406df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.938046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabcd6d6-1e59-4718-bf81-6330284406df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:10.939507 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabcd6d6-1e59-4718-bf81-6330284406df None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-aabcd6d6-1e59-4718-bf81-6330284406df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:10.954407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:10.955734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:10.957058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:10.957058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:10.957058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:10.957058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 640/1251] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:10.962578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae0ed3-3e6f-4ea6-bf11-9e55f0ef0352 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:11.064088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae0ed3-3e6f-4ea6-bf11-9e55f0ef0352 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:11.097177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e135214-63b6-4488-a222-3b286a43251e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:11.097779 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e135214-63b6-4488-a222-3b286a43251e None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:11.098120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e135214-63b6-4488-a222-3b286a43251e None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:11.098394 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e135214-63b6-4488-a222-3b286a43251e None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:11.099037 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e135214-63b6-4488-a222-3b286a43251e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:11.099370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e135214-63b6-4488-a222-3b286a43251e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:11.100766 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e135214-63b6-4488-a222-3b286a43251e None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:11.113399 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 612/1252] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:11.120089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcd6d6-1e59-4718-bf81-6330284406df None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:11.221911 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcd6d6-1e59-4718-bf81-6330284406df None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:11.248203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:11.256877 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa4913-37d7-4b94-9a84-0dcb186769ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:11.257501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa4913-37d7-4b94-9a84-0dcb186769ea None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:11.257859 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa4913-37d7-4b94-9a84-0dcb186769ea None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:11.258137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa4913-37d7-4b94-9a84-0dcb186769ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:11.265136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa4913-37d7-4b94-9a84-0dcb186769ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:11.266578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa4913-37d7-4b94-9a84-0dcb186769ea None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-3cfa4913-37d7-4b94-9a84-0dcb186769ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:11.278256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:11.282767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:11.283957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:11.283957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:11.283957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:11.283957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:11.283957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:11.283957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:11.283957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:11.283957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:11.283957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:35:11.283957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:11.283957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 641/1253] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:11.286653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e135214-63b6-4488-a222-3b286a43251e None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:11.343678 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:11.388423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e135214-63b6-4488-a222-3b286a43251e None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:11.422097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d2222-23b7-4ee3-a23d-9083d0473c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:11.428176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d2222-23b7-4ee3-a23d-9083d0473c17 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:11.428176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d2222-23b7-4ee3-a23d-9083d0473c17 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:11.428176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d2222-23b7-4ee3-a23d-9083d0473c17 None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:11.428176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d2222-23b7-4ee3-a23d-9083d0473c17 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:11.428176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d2222-23b7-4ee3-a23d-9083d0473c17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:11.428176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d2222-23b7-4ee3-a23d-9083d0473c17 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:11.444121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 613/1254] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Aug 26 15:35:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:11.450091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa4913-37d7-4b94-9a84-0dcb186769ea None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:11.550666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa4913-37d7-4b94-9a84-0dcb186769ea None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:11.585178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a26aaf-0df5-40be-aff5-b201a7d8b3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:11.585782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a26aaf-0df5-40be-aff5-b201a7d8b3bf None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:11.586061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a26aaf-0df5-40be-aff5-b201a7d8b3bf None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:11.586335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a26aaf-0df5-40be-aff5-b201a7d8b3bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:11.641990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:11.654029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a26aaf-0df5-40be-aff5-b201a7d8b3bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:11.655483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a26aaf-0df5-40be-aff5-b201a7d8b3bf None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:11.751645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 642/1255] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:35:11.758096 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d2222-23b7-4ee3-a23d-9083d0473c17 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:11.859144 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d2222-23b7-4ee3-a23d-9083d0473c17 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:11.872143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:11.892614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651cd58-ef4a-482d-86a8-be955574d335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:11.893247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651cd58-ef4a-482d-86a8-be955574d335 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:11.893533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651cd58-ef4a-482d-86a8-be955574d335 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:11.893812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651cd58-ef4a-482d-86a8-be955574d335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:11.904292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651cd58-ef4a-482d-86a8-be955574d335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:11.905733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651cd58-ef4a-482d-86a8-be955574d335 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-0651cd58-ef4a-482d-86a8-be955574d335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:11.921212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:11.923782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:11.924917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:11.924917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:11.924917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:11.924917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 614/1256] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:35:11.928822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a26aaf-0df5-40be-aff5-b201a7d8b3bf None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:12.015119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:12.027622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a26aaf-0df5-40be-aff5-b201a7d8b3bf None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:12.060861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b091b-9a29-46d8-aefa-fee93874a957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:12.061467 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b091b-9a29-46d8-aefa-fee93874a957 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:12.061734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b091b-9a29-46d8-aefa-fee93874a957 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:12.062009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b091b-9a29-46d8-aefa-fee93874a957 None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:12.062643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b091b-9a29-46d8-aefa-fee93874a957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:12.062945 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b091b-9a29-46d8-aefa-fee93874a957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:12.064343 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b091b-9a29-46d8-aefa-fee93874a957 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:12.075661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 643/1257] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:12.081807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651cd58-ef4a-482d-86a8-be955574d335 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:12.182582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651cd58-ef4a-482d-86a8-be955574d335 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:12.216997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f808f9-21d6-489d-8c74-cbc3a7adaf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:12.217627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f808f9-21d6-489d-8c74-cbc3a7adaf0b None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:12.217914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f808f9-21d6-489d-8c74-cbc3a7adaf0b None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:12.218193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f808f9-21d6-489d-8c74-cbc3a7adaf0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:12.224795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f808f9-21d6-489d-8c74-cbc3a7adaf0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:12.226293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f808f9-21d6-489d-8c74-cbc3a7adaf0b None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-35f808f9-21d6-489d-8c74-cbc3a7adaf0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:12.237317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:12.238346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:12.239414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:12.239414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:12.239414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:12.239414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:12.239414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:12.239414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:12.239414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:12.239414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:12.239414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 615/1258] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:12.245025 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b091b-9a29-46d8-aefa-fee93874a957 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:12.345523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b091b-9a29-46d8-aefa-fee93874a957 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:12.378047 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c311fb6-0cf2-4243-854f-c586249f02d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:12.387772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c311fb6-0cf2-4243-854f-c586249f02d2 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:12.387772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c311fb6-0cf2-4243-854f-c586249f02d2 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:12.387772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c311fb6-0cf2-4243-854f-c586249f02d2 None admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:12.387772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c311fb6-0cf2-4243-854f-c586249f02d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:12.387772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c311fb6-0cf2-4243-854f-c586249f02d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:12.387772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c311fb6-0cf2-4243-854f-c586249f02d2 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:12.395587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 644/1259] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:12.402362 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f808f9-21d6-489d-8c74-cbc3a7adaf0b None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:12.428419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:12.502808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f808f9-21d6-489d-8c74-cbc3a7adaf0b None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:12.538279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c35d64-8801-4089-86a4-9dd0e3b18cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:12.538885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c35d64-8801-4089-86a4-9dd0e3b18cd6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:12.539169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c35d64-8801-4089-86a4-9dd0e3b18cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:12.539445 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c35d64-8801-4089-86a4-9dd0e3b18cd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:12.540199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c35d64-8801-4089-86a4-9dd0e3b18cd6 None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=b28f987c90df4c19a12bb1819dc65cf9) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:12.540514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c35d64-8801-4089-86a4-9dd0e3b18cd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:12.542212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c35d64-8801-4089-86a4-9dd0e3b18cd6 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:12.549162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 616/1260] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:12 2022] GET /identity/v3/role_assignments?user.id=b28f987c90df4c19a12bb1819dc65cf9&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=6ba5a5add2174983be37258aad1ee404 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:14.438026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1caf2-cf9e-4191-b9fa-ac8b7df2316f None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.438559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1caf2-cf9e-4191-b9fa-ac8b7df2316f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.438559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1caf2-cf9e-4191-b9fa-ac8b7df2316f None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.439948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 645/1261] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:14.445624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05561c09-b125-44d6-b833-92eb2ddd1542 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.446085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05561c09-b125-44d6-b833-92eb2ddd1542 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.446264 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05561c09-b125-44d6-b833-92eb2ddd1542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.464325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-05561c09-b125-44d6-b833-92eb2ddd1542 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:14.465553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-05561c09-b125-44d6-b833-92eb2ddd1542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:14.533122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 617/1262] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:35:14.813210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1caf2-cf9e-4191-b9fa-ac8b7df2316f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:14.824316 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:14.914100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1caf2-cf9e-4191-b9fa-ac8b7df2316f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:14.948230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d4c63-f284-41f5-86d9-774ed228ac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:14.948852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d4c63-f284-41f5-86d9-774ed228ac45 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.949140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d4c63-f284-41f5-86d9-774ed228ac45 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.949466 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d4c63-f284-41f5-86d9-774ed228ac45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.959984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d4c63-f284-41f5-86d9-774ed228ac45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-868d4c63-f284-41f5-86d9-774ed228ac45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:14.964690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:14.965659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:14.967284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 646/1263] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:35:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:35:14.972727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05561c09-b125-44d6-b833-92eb2ddd1542 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:14.993226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:15.071698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05561c09-b125-44d6-b833-92eb2ddd1542 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:15.106821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357e93-9d74-4f7d-baa6-d6ff3d585c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:15.107485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357e93-9d74-4f7d-baa6-d6ff3d585c8c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:15.107768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357e93-9d74-4f7d-baa6-d6ff3d585c8c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:15.108071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357e93-9d74-4f7d-baa6-d6ff3d585c8c None admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:15.108738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94357e93-9d74-4f7d-baa6-d6ff3d585c8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:15.109998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94357e93-9d74-4f7d-baa6-d6ff3d585c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:15.110570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94357e93-9d74-4f7d-baa6-d6ff3d585c8c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:15.118634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 618/1264] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:15.124698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d4c63-f284-41f5-86d9-774ed228ac45 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:15.192641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:15.227148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d4c63-f284-41f5-86d9-774ed228ac45 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:15.261850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c08ea-93c6-4ed7-8089-7fd7b4916f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:15.262485 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c08ea-93c6-4ed7-8089-7fd7b4916f3e None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:15.262751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c08ea-93c6-4ed7-8089-7fd7b4916f3e None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:15.263022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c08ea-93c6-4ed7-8089-7fd7b4916f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:15.274229 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c08ea-93c6-4ed7-8089-7fd7b4916f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:15.275678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c08ea-93c6-4ed7-8089-7fd7b4916f3e None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-483c08ea-93c6-4ed7-8089-7fd7b4916f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:15.289551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:15.290761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:15.292005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:15.292005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:15.292005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:15.292005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 647/1265] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:15.298027 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357e93-9d74-4f7d-baa6-d6ff3d585c8c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:15.379387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:15.399979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357e93-9d74-4f7d-baa6-d6ff3d585c8c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:15.433675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec4a70-3559-4d8c-9faa-370c82809e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:15.434287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec4a70-3559-4d8c-9faa-370c82809e41 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:15.434571 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec4a70-3559-4d8c-9faa-370c82809e41 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:15.434853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec4a70-3559-4d8c-9faa-370c82809e41 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:15.435495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec4a70-3559-4d8c-9faa-370c82809e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:15.435801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec4a70-3559-4d8c-9faa-370c82809e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:15.437349 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec4a70-3559-4d8c-9faa-370c82809e41 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:15.449158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 619/1266] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:15.455895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c08ea-93c6-4ed7-8089-7fd7b4916f3e None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:15.558735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c08ea-93c6-4ed7-8089-7fd7b4916f3e None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:15.579317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:15.593704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8282d03-d4d2-4112-8af3-605ababfc38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:15.594331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8282d03-d4d2-4112-8af3-605ababfc38f None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:15.594620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8282d03-d4d2-4112-8af3-605ababfc38f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:15.594897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8282d03-d4d2-4112-8af3-605ababfc38f None admin] PATH_INFO: `/v3/users/placement` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:15.603019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8282d03-d4d2-4112-8af3-605ababfc38f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:15.604386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8282d03-d4d2-4112-8af3-605ababfc38f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-c8282d03-d4d2-4112-8af3-605ababfc38f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:15.616111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:15.617227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:15.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:15.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:15.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:15.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:15.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:15.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:15.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:15.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:15.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:35:15.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:15.618623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 648/1267] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:15.623998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec4a70-3559-4d8c-9faa-370c82809e41 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:15.751549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec4a70-3559-4d8c-9faa-370c82809e41 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:15.787017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca0544-f5c2-4c8c-9d75-81a7b506ec9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:15.787620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca0544-f5c2-4c8c-9d75-81a7b506ec9c None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:15.787931 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca0544-f5c2-4c8c-9d75-81a7b506ec9c None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:15.788212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca0544-f5c2-4c8c-9d75-81a7b506ec9c None admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:15.789003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca0544-f5c2-4c8c-9d75-81a7b506ec9c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:15.789311 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca0544-f5c2-4c8c-9d75-81a7b506ec9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:15.790800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca0544-f5c2-4c8c-9d75-81a7b506ec9c None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:15.800188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:15.808733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 620/1268] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Aug 26 15:35:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:15.814637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8282d03-d4d2-4112-8af3-605ababfc38f None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:15.915422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8282d03-d4d2-4112-8af3-605ababfc38f None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:15.948411 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b0774-3d33-468f-94d4-8da9fde26fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:15.949023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b0774-3d33-468f-94d4-8da9fde26fab None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:15.949300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b0774-3d33-468f-94d4-8da9fde26fab None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:15.949576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b0774-3d33-468f-94d4-8da9fde26fab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:15.960464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b0774-3d33-468f-94d4-8da9fde26fab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:15.961918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b0774-3d33-468f-94d4-8da9fde26fab None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-cc0b0774-3d33-468f-94d4-8da9fde26fab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:15.976433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:15.978244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:15.980360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:15.980360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:15.980360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:15.980360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 649/1269] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:15.984305 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca0544-f5c2-4c8c-9d75-81a7b506ec9c None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:16.094791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca0544-f5c2-4c8c-9d75-81a7b506ec9c None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:16.129707 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d7407-9e72-4df8-9895-fa32c51ff78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:16.130344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d7407-9e72-4df8-9895-fa32c51ff78a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.131126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d7407-9e72-4df8-9895-fa32c51ff78a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.131401 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d7407-9e72-4df8-9895-fa32c51ff78a None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.132086 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d7407-9e72-4df8-9895-fa32c51ff78a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:16.132391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d7407-9e72-4df8-9895-fa32c51ff78a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:16.133762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d7407-9e72-4df8-9895-fa32c51ff78a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:16.145183 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 621/1270] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:16.151625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b0774-3d33-468f-94d4-8da9fde26fab None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:16.182864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:16.249862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b0774-3d33-468f-94d4-8da9fde26fab None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:16.284701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688f1ac-4d2b-444a-9175-ea9b8db488ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:16.285325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f1ac-4d2b-444a-9175-ea9b8db488ef None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.285662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f1ac-4d2b-444a-9175-ea9b8db488ef None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.285947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f1ac-4d2b-444a-9175-ea9b8db488ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.292895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688f1ac-4d2b-444a-9175-ea9b8db488ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:16.294390 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688f1ac-4d2b-444a-9175-ea9b8db488ef None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-2688f1ac-4d2b-444a-9175-ea9b8db488ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:16.306144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:16.307668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:16.309456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:16.309456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:16.309456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:16.309456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:16.309456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:16.309456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:16.309456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:16.309456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:16.309456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 650/1271] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:16.313658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d7407-9e72-4df8-9895-fa32c51ff78a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:16.373242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:16.414359 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d7407-9e72-4df8-9895-fa32c51ff78a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:16.449107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef740691-5b56-4365-9d5a-3d36661ce3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:16.449720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef740691-5b56-4365-9d5a-3d36661ce3fa None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.450673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef740691-5b56-4365-9d5a-3d36661ce3fa None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.450673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef740691-5b56-4365-9d5a-3d36661ce3fa None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.450949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef740691-5b56-4365-9d5a-3d36661ce3fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:16.451217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef740691-5b56-4365-9d5a-3d36661ce3fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:16.452664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef740691-5b56-4365-9d5a-3d36661ce3fa None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:16.464360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 622/1272] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:16.470474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688f1ac-4d2b-444a-9175-ea9b8db488ef None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:16.564146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:16.573789 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688f1ac-4d2b-444a-9175-ea9b8db488ef None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:16.608741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa45516-1d84-45a6-9206-54b5e56a217f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:16.609416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa45516-1d84-45a6-9206-54b5e56a217f None admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.609682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa45516-1d84-45a6-9206-54b5e56a217f None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.609960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa45516-1d84-45a6-9206-54b5e56a217f None admin] PATH_INFO: `/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/b28f987c90df4c19a12bb1819dc65cf9/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.611829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa45516-1d84-45a6-9206-54b5e56a217f None admin] RBAC: Authorizing `identity:create_grant(project_id=19b186f6269a4406b3fc499a582a6e66, user_id=b28f987c90df4c19a12bb1819dc65cf9, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:16.613379 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa45516-1d84-45a6-9206-54b5e56a217f None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:16.789338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 651/1273] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Aug 26 15:35:16 2022] PUT /identity/v3/projects/19b186f6269a4406b3fc499a582a6e66/users/b28f987c90df4c19a12bb1819dc65cf9/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:35:18.709222 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab00e6-febe-4b82-bea6-6a6a5e2d3cc7 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:18.709901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab00e6-febe-4b82-bea6-6a6a5e2d3cc7 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:18.709901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab00e6-febe-4b82-bea6-6a6a5e2d3cc7 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:18.711235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 623/1274] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:18.716557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a510-99e2-4858-96db-d1235a6efc53 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:18.716961 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a510-99e2-4858-96db-d1235a6efc53 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:18.717148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a510-99e2-4858-96db-d1235a6efc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:18.736119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-4f21a510-99e2-4858-96db-d1235a6efc53 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:18.737352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4f21a510-99e2-4858-96db-d1235a6efc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:18.807881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 652/1275] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:35:19.094913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab00e6-febe-4b82-bea6-6a6a5e2d3cc7 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:19.209125 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab00e6-febe-4b82-bea6-6a6a5e2d3cc7 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:19.227365 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:19.243478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e76361-cb07-4b57-8bf4-f9102da6f3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:19.244125 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e76361-cb07-4b57-8bf4-f9102da6f3d6 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:19.244409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e76361-cb07-4b57-8bf4-f9102da6f3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:19.244683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e76361-cb07-4b57-8bf4-f9102da6f3d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:19.254306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e76361-cb07-4b57-8bf4-f9102da6f3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-27e76361-cb07-4b57-8bf4-f9102da6f3d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:19.259428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:19.266063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:19.267131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 624/1276] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Aug 26 15:35:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:19.268030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21a510-99e2-4858-96db-d1235a6efc53 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:19.345760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:19.363210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21a510-99e2-4858-96db-d1235a6efc53 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:19.414118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35687b53-7a75-4f14-b87b-25fe10b90421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:19.414737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687b53-7a75-4f14-b87b-25fe10b90421 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:19.415025 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687b53-7a75-4f14-b87b-25fe10b90421 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:19.415305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687b53-7a75-4f14-b87b-25fe10b90421 None admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:19.416054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35687b53-7a75-4f14-b87b-25fe10b90421 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:19.416633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35687b53-7a75-4f14-b87b-25fe10b90421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:19.418023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35687b53-7a75-4f14-b87b-25fe10b90421 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:19.427087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 653/1277] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:19.433028 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e76361-cb07-4b57-8bf4-f9102da6f3d6 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:19.534518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e76361-cb07-4b57-8bf4-f9102da6f3d6 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:19.572403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3b8bf-01a4-409f-b651-e6e5a5f93acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:19.573042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3b8bf-01a4-409f-b651-e6e5a5f93acc None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:19.573416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3b8bf-01a4-409f-b651-e6e5a5f93acc None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:19.573697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3b8bf-01a4-409f-b651-e6e5a5f93acc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:19.584272 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3b8bf-01a4-409f-b651-e6e5a5f93acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:19.585744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3b8bf-01a4-409f-b651-e6e5a5f93acc None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-6ba3b8bf-01a4-409f-b651-e6e5a5f93acc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:19.610570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:19.611726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:19.612973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:19.612973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:19.612973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:19.612973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 625/1278] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:35:19.613265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35687b53-7a75-4f14-b87b-25fe10b90421 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:19.709390 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35687b53-7a75-4f14-b87b-25fe10b90421 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:19.743489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794152aa-79f7-4047-adb9-16e101c27f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:19.744212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794152aa-79f7-4047-adb9-16e101c27f3c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:19.744395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794152aa-79f7-4047-adb9-16e101c27f3c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:19.744670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794152aa-79f7-4047-adb9-16e101c27f3c None admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:19.745309 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794152aa-79f7-4047-adb9-16e101c27f3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:19.745611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794152aa-79f7-4047-adb9-16e101c27f3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:19.746973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794152aa-79f7-4047-adb9-16e101c27f3c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:19.758831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 654/1279] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:19.764771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3b8bf-01a4-409f-b651-e6e5a5f93acc None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:19.796326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:19.865299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3b8bf-01a4-409f-b651-e6e5a5f93acc None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:19.901526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc88eb-6009-4c75-84c8-18df6fd388e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:19.902158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc88eb-6009-4c75-84c8-18df6fd388e2 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:19.902443 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc88eb-6009-4c75-84c8-18df6fd388e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:19.902723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc88eb-6009-4c75-84c8-18df6fd388e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:19.909851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc88eb-6009-4c75-84c8-18df6fd388e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:19.911306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc88eb-6009-4c75-84c8-18df6fd388e2 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-78bc88eb-6009-4c75-84c8-18df6fd388e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:19.926645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:19.928961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:19.930173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:19.930173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:19.930173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:19.930173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:19.930173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:19.930173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:19.930173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:19.930173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:19.930173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:35:19.930173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:19.930173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 626/1280] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:19.933713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794152aa-79f7-4047-adb9-16e101c27f3c None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:19.937579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:20.035018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794152aa-79f7-4047-adb9-16e101c27f3c None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:20.068826 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c58b3-1d9b-4c0d-8959-36a7319ebab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:20.069428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c58b3-1d9b-4c0d-8959-36a7319ebab2 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.069715 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c58b3-1d9b-4c0d-8959-36a7319ebab2 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.069994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c58b3-1d9b-4c0d-8959-36a7319ebab2 None admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.070720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c58b3-1d9b-4c0d-8959-36a7319ebab2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:20.071027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c58b3-1d9b-4c0d-8959-36a7319ebab2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:20.072486 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c58b3-1d9b-4c0d-8959-36a7319ebab2 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:20.089121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 655/1281] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Aug 26 15:35:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:20.094932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc88eb-6009-4c75-84c8-18df6fd388e2 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:20.188359 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:20.200686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc88eb-6009-4c75-84c8-18df6fd388e2 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:20.234003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add74aa0-9435-478a-9ead-b69c52fdd155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:20.234609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add74aa0-9435-478a-9ead-b69c52fdd155 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.234891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add74aa0-9435-478a-9ead-b69c52fdd155 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.235169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add74aa0-9435-478a-9ead-b69c52fdd155 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.301262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add74aa0-9435-478a-9ead-b69c52fdd155 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:20.303227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add74aa0-9435-478a-9ead-b69c52fdd155 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:20.390988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:20.400950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 627/1282] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:35:20.406983 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c58b3-1d9b-4c0d-8959-36a7319ebab2 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:20.473867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:20.513189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c58b3-1d9b-4c0d-8959-36a7319ebab2 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:20.545862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc652ba-cd4c-416d-a294-e41941cd4eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:20.546520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc652ba-cd4c-416d-a294-e41941cd4eb6 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.546807 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc652ba-cd4c-416d-a294-e41941cd4eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.547089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc652ba-cd4c-416d-a294-e41941cd4eb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.557997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc652ba-cd4c-416d-a294-e41941cd4eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:20.562162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc652ba-cd4c-416d-a294-e41941cd4eb6 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-4fc652ba-cd4c-416d-a294-e41941cd4eb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:20.573086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:20.574277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:20.575702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:20.575702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:20.575702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:20.575702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 656/1283] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:20.580099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add74aa0-9435-478a-9ead-b69c52fdd155 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:20.684754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add74aa0-9435-478a-9ead-b69c52fdd155 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:20.717689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2aa3e-08ed-4960-a3df-b8e013a66fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:20.718297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2aa3e-08ed-4960-a3df-b8e013a66fc9 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.718584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2aa3e-08ed-4960-a3df-b8e013a66fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.718865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2aa3e-08ed-4960-a3df-b8e013a66fc9 None admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.719513 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2aa3e-08ed-4960-a3df-b8e013a66fc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:20.719819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2aa3e-08ed-4960-a3df-b8e013a66fc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:20.721234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2aa3e-08ed-4960-a3df-b8e013a66fc9 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:20.733065 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 628/1284] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:20.738885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc652ba-cd4c-416d-a294-e41941cd4eb6 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:20.864427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc652ba-cd4c-416d-a294-e41941cd4eb6 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:20.878455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:20.898894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0655dfe-22d5-48a2-86d5-70a9c297223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:20.899526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0655dfe-22d5-48a2-86d5-70a9c297223d None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.899810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0655dfe-22d5-48a2-86d5-70a9c297223d None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.900115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0655dfe-22d5-48a2-86d5-70a9c297223d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.907275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0655dfe-22d5-48a2-86d5-70a9c297223d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:20.908775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0655dfe-22d5-48a2-86d5-70a9c297223d None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application [None req-d0655dfe-22d5-48a2-86d5-70a9c297223d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:20.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:20.927286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:20.928477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:20.928477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:20.928477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:20.928477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:20.928477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:20.928477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:20.928477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:20.928477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: ERROR keystone.server.flask.application Aug 26 15:35:20.928477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 657/1285] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:20.931708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2aa3e-08ed-4960-a3df-b8e013a66fc9 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:20.984586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:21.038241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2aa3e-08ed-4960-a3df-b8e013a66fc9 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:21.076893 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1f8e9-15e3-453e-bebf-760f2b376a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:21.077515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f8e9-15e3-453e-bebf-760f2b376a5a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.077798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f8e9-15e3-453e-bebf-760f2b376a5a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.078074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f8e9-15e3-453e-bebf-760f2b376a5a None admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.078827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1f8e9-15e3-453e-bebf-760f2b376a5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:21.079171 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1f8e9-15e3-453e-bebf-760f2b376a5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:21.080757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1f8e9-15e3-453e-bebf-760f2b376a5a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:21.094192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 629/1286] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:21.101194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0655dfe-22d5-48a2-86d5-70a9c297223d None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:21.203778 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0655dfe-22d5-48a2-86d5-70a9c297223d None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:21.238064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0834147-647a-406f-944c-81a5207134df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:21.238596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0834147-647a-406f-944c-81a5207134df None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.238885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0834147-647a-406f-944c-81a5207134df None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.239166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0834147-647a-406f-944c-81a5207134df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.239943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0834147-647a-406f-944c-81a5207134df None admin] RBAC: Adding query filter params (role.id=6ba5a5add2174983be37258aad1ee404, scope.project.id=19b186f6269a4406b3fc499a582a6e66, user.id=b28f987c90df4c19a12bb1819dc65cf9) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:21.240260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0834147-647a-406f-944c-81a5207134df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:21.241747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0834147-647a-406f-944c-81a5207134df None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:21.247430 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 658/1287] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:21 2022] GET /identity/v3/role_assignments?user.id=b28f987c90df4c19a12bb1819dc65cf9&scope.project.id=19b186f6269a4406b3fc499a582a6e66&role.id=6ba5a5add2174983be37258aad1ee404 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:23.202371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ffcef-ea45-4a69-910e-25be5fa18a20 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:23.202953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ffcef-ea45-4a69-910e-25be5fa18a20 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:23.203097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ffcef-ea45-4a69-910e-25be5fa18a20 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:23.204772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 630/1288] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:35:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:23.210144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8f889-eebb-4993-9ce8-0f10c9117416 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:23.210553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8f889-eebb-4993-9ce8-0f10c9117416 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:23.210772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8f889-eebb-4993-9ce8-0f10c9117416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:23.228690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-47d8f889-eebb-4993-9ce8-0f10c9117416 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:23.229881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-47d8f889-eebb-4993-9ce8-0f10c9117416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:23.260383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:23.295977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 659/1289] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:35:23.575747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ffcef-ea45-4a69-910e-25be5fa18a20 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:23.681085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ffcef-ea45-4a69-910e-25be5fa18a20 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:23.692154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:23.715049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1be53f-b2ad-42e0-8c6f-be370d8ae973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:23.720943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be53f-b2ad-42e0-8c6f-be370d8ae973 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:23.720943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be53f-b2ad-42e0-8c6f-be370d8ae973 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:23.720943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be53f-b2ad-42e0-8c6f-be370d8ae973 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:23.720943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1be53f-b2ad-42e0-8c6f-be370d8ae973 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:23.720943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1be53f-b2ad-42e0-8c6f-be370d8ae973 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-ee1be53f-b2ad-42e0-8c6f-be370d8ae973 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:23.727465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:23.728630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:23.729690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 631/1290] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Aug 26 15:35:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:23.736490 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8f889-eebb-4993-9ce8-0f10c9117416 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:23.847974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8f889-eebb-4993-9ce8-0f10c9117416 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:23.884411 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de9bd6-e5e3-4746-9753-9a93875d68d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:23.885030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de9bd6-e5e3-4746-9753-9a93875d68d9 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:23.885316 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de9bd6-e5e3-4746-9753-9a93875d68d9 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:23.885594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de9bd6-e5e3-4746-9753-9a93875d68d9 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:23.886234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de9bd6-e5e3-4746-9753-9a93875d68d9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:23.886540 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de9bd6-e5e3-4746-9753-9a93875d68d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:23.887959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de9bd6-e5e3-4746-9753-9a93875d68d9 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:23.893054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 660/1291] 10.222.0.98 () {56 vars in 1116 bytes} [Fri Aug 26 15:35:23 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:23.900263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1be53f-b2ad-42e0-8c6f-be370d8ae973 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:24.001680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1be53f-b2ad-42e0-8c6f-be370d8ae973 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:24.035415 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d95dc5-efec-44bb-aeae-871c01e96ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:24.036087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d95dc5-efec-44bb-aeae-871c01e96ba1 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:24.036371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d95dc5-efec-44bb-aeae-871c01e96ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:24.037205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d95dc5-efec-44bb-aeae-871c01e96ba1 None admin] PATH_INFO: `/v3/services` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:24.037836 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d95dc5-efec-44bb-aeae-871c01e96ba1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:24.038141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d95dc5-efec-44bb-aeae-871c01e96ba1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:24.039503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d95dc5-efec-44bb-aeae-871c01e96ba1 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:24.044205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 632/1292] 10.222.0.98 () {56 vars in 1116 bytes} [Fri Aug 26 15:35:23 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:25.959117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65627407-84d3-4c92-87ca-4a1909ca559b None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:25.959666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65627407-84d3-4c92-87ca-4a1909ca559b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:25.959666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65627407-84d3-4c92-87ca-4a1909ca559b None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:25.961052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 661/1293] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:35:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:25.966442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9778d-0a19-414a-bf16-0321a17b9f02 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:25.971679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9778d-0a19-414a-bf16-0321a17b9f02 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:25.971679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9778d-0a19-414a-bf16-0321a17b9f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:25.984651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e0c9778d-0a19-414a-bf16-0321a17b9f02 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:25.985855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e0c9778d-0a19-414a-bf16-0321a17b9f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:26.015188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:26.051645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 633/1294] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:35:26.323990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65627407-84d3-4c92-87ca-4a1909ca559b None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:26.355003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:26.425226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65627407-84d3-4c92-87ca-4a1909ca559b None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:26.458779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb7d0a-5223-480e-ba1c-5bfd0785dcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:26.459375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb7d0a-5223-480e-ba1c-5bfd0785dcd2 None admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:26.459662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb7d0a-5223-480e-ba1c-5bfd0785dcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:26.459967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb7d0a-5223-480e-ba1c-5bfd0785dcd2 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:26.460620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb7d0a-5223-480e-ba1c-5bfd0785dcd2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:26.461987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb7d0a-5223-480e-ba1c-5bfd0785dcd2 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:26.597604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 662/1295] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Aug 26 15:35:26 2022] POST /identity/v3/services => generated 246 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:35:28.505140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f91f64-f8f5-4dcf-8fed-62014fe5c000 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.505720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f91f64-f8f5-4dcf-8fed-62014fe5c000 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.505720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f91f64-f8f5-4dcf-8fed-62014fe5c000 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.507070 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 634/1296] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:28.512446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14ca2-9071-4bd5-9362-28277919a747 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.512848 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14ca2-9071-4bd5-9362-28277919a747 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.512997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14ca2-9071-4bd5-9362-28277919a747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.531207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e4e14ca2-9071-4bd5-9362-28277919a747 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:28.532445 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e4e14ca2-9071-4bd5-9362-28277919a747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:28.607069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 663/1297] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:28 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:35:28.876390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f91f64-f8f5-4dcf-8fed-62014fe5c000 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:28.978291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f91f64-f8f5-4dcf-8fed-62014fe5c000 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.011431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97313b1-295c-408a-be33-5cfdfc912f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.012088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97313b1-295c-408a-be33-5cfdfc912f05 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.012371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97313b1-295c-408a-be33-5cfdfc912f05 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.012650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97313b1-295c-408a-be33-5cfdfc912f05 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.013233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97313b1-295c-408a-be33-5cfdfc912f05 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:29.014596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97313b1-295c-408a-be33-5cfdfc912f05 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-c97313b1-295c-408a-be33-5cfdfc912f05 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:29.023698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:29.024925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:29.025826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 635/1298] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Aug 26 15:35:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:29.030851 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e14ca2-9071-4bd5-9362-28277919a747 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:29.049476 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:29.126592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e14ca2-9071-4bd5-9362-28277919a747 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.159088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444889dc-c7cd-411f-bf40-95e428cb4603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.162245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444889dc-c7cd-411f-bf40-95e428cb4603 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.162245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444889dc-c7cd-411f-bf40-95e428cb4603 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.162245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444889dc-c7cd-411f-bf40-95e428cb4603 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.162245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444889dc-c7cd-411f-bf40-95e428cb4603 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:29.162245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444889dc-c7cd-411f-bf40-95e428cb4603 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:29.162704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444889dc-c7cd-411f-bf40-95e428cb4603 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:29.168425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 664/1299] 10.222.0.98 () {56 vars in 1116 bytes} [Fri Aug 26 15:35:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:29.174382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97313b1-295c-408a-be33-5cfdfc912f05 None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:29.205505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:29.271155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97313b1-295c-408a-be33-5cfdfc912f05 None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.305646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d892ac-fb57-40b5-8b09-059d1d32f8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.306271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d892ac-fb57-40b5-8b09-059d1d32f8c5 None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.306551 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d892ac-fb57-40b5-8b09-059d1d32f8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.306821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d892ac-fb57-40b5-8b09-059d1d32f8c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.307510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d892ac-fb57-40b5-8b09-059d1d32f8c5 None admin] RBAC: Adding query filter params (interface=public, service_id=0091a1ed9dcd4bbea8fa35f24066ee65, region_id=RegionOne) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:29.307814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d892ac-fb57-40b5-8b09-059d1d32f8c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:29.309193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d892ac-fb57-40b5-8b09-059d1d32f8c5 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:29.314279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 636/1300] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Aug 26 15:35:29 2022] GET /identity/v3/endpoints?service_id=0091a1ed9dcd4bbea8fa35f24066ee65&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:29.320121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444889dc-c7cd-411f-bf40-95e428cb4603 None admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:29.400356 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:29.417783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444889dc-c7cd-411f-bf40-95e428cb4603 None admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.451764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a643f-c87a-48f7-a8ec-57588a51f82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.452397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a643f-c87a-48f7-a8ec-57588a51f82c None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.452686 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a643f-c87a-48f7-a8ec-57588a51f82c None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.452965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a643f-c87a-48f7-a8ec-57588a51f82c None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.453566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a643f-c87a-48f7-a8ec-57588a51f82c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:29.454923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a643f-c87a-48f7-a8ec-57588a51f82c None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:29.462077 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 665/1301] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Aug 26 15:35:29 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:35:31.386944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054821d2-c7b0-4150-a69c-f374cc733e07 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.387781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054821d2-c7b0-4150-a69c-f374cc733e07 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.387992 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054821d2-c7b0-4150-a69c-f374cc733e07 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.389342 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 637/1302] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:31.394447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532d2f-b2ed-4f68-aff6-c4a7d0ca96c7 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.394849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532d2f-b2ed-4f68-aff6-c4a7d0ca96c7 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.395011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532d2f-b2ed-4f68-aff6-c4a7d0ca96c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.413357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-67532d2f-b2ed-4f68-aff6-c4a7d0ca96c7 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:31.414541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-67532d2f-b2ed-4f68-aff6-c4a7d0ca96c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:31.479112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 666/1303] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:35:31.766418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054821d2-c7b0-4150-a69c-f374cc733e07 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:31.860631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:31.867841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054821d2-c7b0-4150-a69c-f374cc733e07 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:31.901340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d2959-6926-456e-a202-56ca9e8f517a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:31.901964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d2959-6926-456e-a202-56ca9e8f517a None admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.902253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d2959-6926-456e-a202-56ca9e8f517a None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.902532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d2959-6926-456e-a202-56ca9e8f517a None admin] PATH_INFO: `/v3/services/placement` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.903110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d2959-6926-456e-a202-56ca9e8f517a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:31.904502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d2959-6926-456e-a202-56ca9e8f517a None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application [None req-af0d2959-6926-456e-a202-56ca9e8f517a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:31.913722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:31.915060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: ERROR keystone.server.flask.application Aug 26 15:35:31.916167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 638/1304] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Aug 26 15:35:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:31.921419 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67532d2f-b2ed-4f68-aff6-c4a7d0ca96c7 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:31.999080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:32.020686 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67532d2f-b2ed-4f68-aff6-c4a7d0ca96c7 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:32.053654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933abce4-d0d7-4999-98ed-292a96a7d718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:32.054449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933abce4-d0d7-4999-98ed-292a96a7d718 None admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:32.054567 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933abce4-d0d7-4999-98ed-292a96a7d718 None admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:32.054777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933abce4-d0d7-4999-98ed-292a96a7d718 None admin] PATH_INFO: `/v3/services` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:32.055397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933abce4-d0d7-4999-98ed-292a96a7d718 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:32.055714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933abce4-d0d7-4999-98ed-292a96a7d718 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:32.057096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933abce4-d0d7-4999-98ed-292a96a7d718 None admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:32.062024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 667/1305] 10.222.0.98 () {56 vars in 1116 bytes} [Fri Aug 26 15:35:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:32.067938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d2959-6926-456e-a202-56ca9e8f517a None admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:32.166928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d2959-6926-456e-a202-56ca9e8f517a None admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:32.198705 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4add7c1-b507-47b8-85ef-0ed813adc755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:32.199302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4add7c1-b507-47b8-85ef-0ed813adc755 None admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:32.199589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4add7c1-b507-47b8-85ef-0ed813adc755 None admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:32.199893 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4add7c1-b507-47b8-85ef-0ed813adc755 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:32.200563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4add7c1-b507-47b8-85ef-0ed813adc755 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:32.201969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4add7c1-b507-47b8-85ef-0ed813adc755 None admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:32.305695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 639/1306] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:35:32 2022] POST /identity/v3/endpoints => generated 331 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:35:42.071547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=30 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:42.300662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:34.188138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f54eb-48a2-485e-9dbb-d362dddca8d4 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:34.189247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f54eb-48a2-485e-9dbb-d362dddca8d4 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:34.189758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f54eb-48a2-485e-9dbb-d362dddca8d4 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:34.196992 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1204920-1261-4055-8e9a-fde8ed5d05d6 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:34.197947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1204920-1261-4055-8e9a-fde8ed5d05d6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:34.198449 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1204920-1261-4055-8e9a-fde8ed5d05d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:34.201583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 668/1307] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:41:34 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:41:34.289493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b1204920-1261-4055-8e9a-fde8ed5d05d6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:41:34.291092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b1204920-1261-4055-8e9a-fde8ed5d05d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:34.543892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 640/1308] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:41:34.856840 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423c049-170c-4d00-9e60-0c786354cdf6 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:34.857505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423c049-170c-4d00-9e60-0c786354cdf6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:34.857999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423c049-170c-4d00-9e60-0c786354cdf6 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:34.860195 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 669/1309] 10.222.0.98 () {54 vars in 965 bytes} [Fri Aug 26 15:41:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:41:34.866941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622a8d2-2727-4939-9893-1e211e023d38 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:34.867241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622a8d2-2727-4939-9893-1e211e023d38 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:34.867528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622a8d2-2727-4939-9893-1e211e023d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:34.954421 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-1622a8d2-2727-4939-9893-1e211e023d38 None None] MFA Rules not processed for user `8c6cad18359143868276eb9d79a3fd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:41:34.955655 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-1622a8d2-2727-4939-9893-1e211e023d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:34.979349 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:35.049509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 641/1310] 10.222.0.98 () {58 vars in 1062 bytes} [Fri Aug 26 15:41:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:41:35.059374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423c049-170c-4d00-9e60-0c786354cdf6 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:41:35.061738 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3423c049-170c-4d00-9e60-0c786354cdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:35.192797 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423c049-170c-4d00-9e60-0c786354cdf6 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:41:35.273213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87430c-1ad4-4aa8-8106-99e33f40eef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:41:35.274241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87430c-1ad4-4aa8-8106-99e33f40eef1 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:35.274784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87430c-1ad4-4aa8-8106-99e33f40eef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:35.275280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87430c-1ad4-4aa8-8106-99e33f40eef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:35.277370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6c87430c-1ad4-4aa8-8106-99e33f40eef1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:35.338180 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:35.399849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87430c-1ad4-4aa8-8106-99e33f40eef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:41:35.402022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87430c-1ad4-4aa8-8106-99e33f40eef1 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:41:35.425214 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 670/1311] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:41:37.714781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8b9d3-6e78-4061-b241-4250b7f78e69 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:37.715325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8b9d3-6e78-4061-b241-4250b7f78e69 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:37.715325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8b9d3-6e78-4061-b241-4250b7f78e69 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:37.716743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 642/1312] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:41:37.722661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5fd07-c402-4858-8f22-84c7d3bbeeae None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:37.723182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5fd07-c402-4858-8f22-84c7d3bbeeae None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:37.723276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5fd07-c402-4858-8f22-84c7d3bbeeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:37.803095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-82e5fd07-c402-4858-8f22-84c7d3bbeeae None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:41:37.804824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-82e5fd07-c402-4858-8f22-84c7d3bbeeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:37.915614 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 671/1313] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:41:38.211019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68175-cd31-4093-87fc-8686cc3fb8a9 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:38.211310 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68175-cd31-4093-87fc-8686cc3fb8a9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:38.211594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68175-cd31-4093-87fc-8686cc3fb8a9 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:38.212986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 643/1314] 10.222.0.98 () {54 vars in 965 bytes} [Fri Aug 26 15:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:41:38.218873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7bec9-e05d-4b06-99cf-41484964d5d6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:38.219170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7bec9-e05d-4b06-99cf-41484964d5d6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:38.219446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7bec9-e05d-4b06-99cf-41484964d5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:38.231709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:38.302947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-99f7bec9-e05d-4b06-99cf-41484964d5d6 None None] MFA Rules not processed for user `8c6cad18359143868276eb9d79a3fd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:41:38.304978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-99f7bec9-e05d-4b06-99cf-41484964d5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:38.398217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 672/1315] 10.222.0.98 () {58 vars in 1062 bytes} [Fri Aug 26 15:41:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:41:38.407037 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d68175-cd31-4093-87fc-8686cc3fb8a9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:41:38.408682 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-04d68175-cd31-4093-87fc-8686cc3fb8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:38.512887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d68175-cd31-4093-87fc-8686cc3fb8a9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:41:38.585122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:38.587751 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a8b10-1b12-4b9f-bd52-1bdc52810f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:41:38.588416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a8b10-1b12-4b9f-bd52-1bdc52810f66 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:38.588758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a8b10-1b12-4b9f-bd52-1bdc52810f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:38.589070 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a8b10-1b12-4b9f-bd52-1bdc52810f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:38.590739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0e3a8b10-1b12-4b9f-bd52-1bdc52810f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:38.707124 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a8b10-1b12-4b9f-bd52-1bdc52810f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:41:38.708595 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a8b10-1b12-4b9f-bd52-1bdc52810f66 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:41:38.726893 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 644/1316] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:41:41.380705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ae6aa-4141-4fcc-b67b-19ce231d9394 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:41.381240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ae6aa-4141-4fcc-b67b-19ce231d9394 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:41.381352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ae6aa-4141-4fcc-b67b-19ce231d9394 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:41.382669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 673/1317] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:41:41.388328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc790a2-981d-4fd2-90af-841d9e3416e4 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:41.388791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc790a2-981d-4fd2-90af-841d9e3416e4 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:41.388885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc790a2-981d-4fd2-90af-841d9e3416e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:41.459398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-bcc790a2-981d-4fd2-90af-841d9e3416e4 None None] MFA Rules not processed for user `99da68de463743fdb9293f16abb81a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:41:41.460726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-bcc790a2-981d-4fd2-90af-841d9e3416e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:41.516192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:41.566186 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 645/1318] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:41:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:41:41.855889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ae6aa-4141-4fcc-b67b-19ce231d9394 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:41:41.857450 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-318ae6aa-4141-4fcc-b67b-19ce231d9394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:41.964936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ae6aa-4141-4fcc-b67b-19ce231d9394 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:41:41.967706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:42.043109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464d8d9-797f-46b1-85ff-c99eda29c9fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:41:42.043719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464d8d9-797f-46b1-85ff-c99eda29c9fe service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:42.044037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464d8d9-797f-46b1-85ff-c99eda29c9fe service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:42.044315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464d8d9-797f-46b1-85ff-c99eda29c9fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:42.046087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4464d8d9-797f-46b1-85ff-c99eda29c9fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:42.153698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464d8d9-797f-46b1-85ff-c99eda29c9fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:41:42.155032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464d8d9-797f-46b1-85ff-c99eda29c9fe service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:41:42.173143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 674/1319] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:41:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:41:45.005081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaefede-1bae-4822-b27d-12488c55e401 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:45.005649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaefede-1bae-4822-b27d-12488c55e401 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:45.005649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaefede-1bae-4822-b27d-12488c55e401 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:45.006963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 646/1320] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:41:45.012510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1068d-ffee-4508-96f3-b48adc883398 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:45.012982 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1068d-ffee-4508-96f3-b48adc883398 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:45.013079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1068d-ffee-4508-96f3-b48adc883398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:45.056878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:45.083301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-67c1068d-ffee-4508-96f3-b48adc883398 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:41:45.084566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-67c1068d-ffee-4508-96f3-b48adc883398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:45.186133 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 675/1321] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:41:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:41:45.469805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaefede-1bae-4822-b27d-12488c55e401 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:41:45.471406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-eaaefede-1bae-4822-b27d-12488c55e401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:45.578157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaefede-1bae-4822-b27d-12488c55e401 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:41:45.655130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d6471-3a79-4e81-9a69-f55c7f39a78f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:41:45.656320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d6471-3a79-4e81-9a69-f55c7f39a78f service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:45.656606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d6471-3a79-4e81-9a69-f55c7f39a78f service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:45.656883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d6471-3a79-4e81-9a69-f55c7f39a78f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:45.658555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-408d6471-3a79-4e81-9a69-f55c7f39a78f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:45.666317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:45.778874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d6471-3a79-4e81-9a69-f55c7f39a78f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:41:45.780266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d6471-3a79-4e81-9a69-f55c7f39a78f service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:41:45.799335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 647/1322] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:41:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:41:48.946768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284eb1b-8b92-4cba-9000-33776dfc759d None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:48.947368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284eb1b-8b92-4cba-9000-33776dfc759d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:48.947368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284eb1b-8b92-4cba-9000-33776dfc759d None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:48.948873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 676/1323] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:41:48.954400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7f43e-4893-448e-ad90-b3db3709cd86 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:48.954857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7f43e-4893-448e-ad90-b3db3709cd86 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:48.954951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7f43e-4893-448e-ad90-b3db3709cd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:49.026058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-3af7f43e-4893-448e-ad90-b3db3709cd86 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:41:49.027290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3af7f43e-4893-448e-ad90-b3db3709cd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:49.112796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:49.157085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 648/1324] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:41:49.439004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284eb1b-8b92-4cba-9000-33776dfc759d None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:41:49.440801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e284eb1b-8b92-4cba-9000-33776dfc759d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:49.561825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284eb1b-8b92-4cba-9000-33776dfc759d None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:41:49.579139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:49.653342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65376148-1a9d-4359-9507-946b8e722302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:41:49.653949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65376148-1a9d-4359-9507-946b8e722302 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:49.654231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65376148-1a9d-4359-9507-946b8e722302 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:49.654508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65376148-1a9d-4359-9507-946b8e722302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:49.656177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-65376148-1a9d-4359-9507-946b8e722302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:49.777477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65376148-1a9d-4359-9507-946b8e722302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:41:49.778857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65376148-1a9d-4359-9507-946b8e722302 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:41:49.798172 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 677/1325] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:41:51.775069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe66880-f9bb-47d1-a591-3ca73e940118 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:51.775636 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe66880-f9bb-47d1-a591-3ca73e940118 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:51.775636 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe66880-f9bb-47d1-a591-3ca73e940118 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:51.776990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 649/1326] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:41:51.782860 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b6bd9-2aa7-4ffa-9678-c7b0f451b00c None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:51.783329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b6bd9-2aa7-4ffa-9678-c7b0f451b00c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:51.783545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b6bd9-2aa7-4ffa-9678-c7b0f451b00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:51.832556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:51.850566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-fe5b6bd9-2aa7-4ffa-9678-c7b0f451b00c None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:41:51.851778 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-fe5b6bd9-2aa7-4ffa-9678-c7b0f451b00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:51.949267 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 678/1327] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:41:52.231460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe66880-f9bb-47d1-a591-3ca73e940118 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:41:52.233555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ffe66880-f9bb-47d1-a591-3ca73e940118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:52.342033 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe66880-f9bb-47d1-a591-3ca73e940118 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:41:52.424303 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee0f07-c206-4e4f-8781-560f7e805639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:41:52.424914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee0f07-c206-4e4f-8781-560f7e805639 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:52.425194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee0f07-c206-4e4f-8781-560f7e805639 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:52.425728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee0f07-c206-4e4f-8781-560f7e805639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:52.427412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c3ee0f07-c206-4e4f-8781-560f7e805639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:52.445299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:52.539667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee0f07-c206-4e4f-8781-560f7e805639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:41:52.541158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee0f07-c206-4e4f-8781-560f7e805639 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:41:52.561078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 650/1328] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:41:54.564622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdabeb-0d77-4e53-a7c7-eeb53c5f443e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:54.565178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdabeb-0d77-4e53-a7c7-eeb53c5f443e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:54.565178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdabeb-0d77-4e53-a7c7-eeb53c5f443e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:54.566499 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 679/1329] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:41:54.572163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a3aae-f9da-4637-99ad-428dcc7baab1 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:54.572704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a3aae-f9da-4637-99ad-428dcc7baab1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:54.572852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a3aae-f9da-4637-99ad-428dcc7baab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:54.647966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-dc0a3aae-f9da-4637-99ad-428dcc7baab1 None None] MFA Rules not processed for user `99da68de463743fdb9293f16abb81a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:41:54.649408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-dc0a3aae-f9da-4637-99ad-428dcc7baab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:54.728069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:54.764357 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 651/1330] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:41:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:41:55.048196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdabeb-0d77-4e53-a7c7-eeb53c5f443e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:41:55.049929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-defdabeb-0d77-4e53-a7c7-eeb53c5f443e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:55.156151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdabeb-0d77-4e53-a7c7-eeb53c5f443e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:41:55.176303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:55.245829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b1cf1-9483-4f95-b1f8-67dfe6f52643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:41:55.246514 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b1cf1-9483-4f95-b1f8-67dfe6f52643 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:55.246829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b1cf1-9483-4f95-b1f8-67dfe6f52643 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:55.247106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b1cf1-9483-4f95-b1f8-67dfe6f52643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:55.248970 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-119b1cf1-9483-4f95-b1f8-67dfe6f52643 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:55.369973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b1cf1-9483-4f95-b1f8-67dfe6f52643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:41:55.371492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b1cf1-9483-4f95-b1f8-67dfe6f52643 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:41:55.390421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 680/1331] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:41:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:41:58.155089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406c905-9248-4695-8de9-e0cd8f3a09ee None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:58.155695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406c905-9248-4695-8de9-e0cd8f3a09ee None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:58.155695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406c905-9248-4695-8de9-e0cd8f3a09ee None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:58.157033 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 652/1332] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:41:58.162625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227b5e5-b639-4451-bd52-586fab9397c3 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:58.163053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227b5e5-b639-4451-bd52-586fab9397c3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:58.163192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227b5e5-b639-4451-bd52-586fab9397c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:58.228279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:58.231196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0227b5e5-b639-4451-bd52-586fab9397c3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:41:58.232449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0227b5e5-b639-4451-bd52-586fab9397c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:58.330563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 681/1333] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:41:58.613241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406c905-9248-4695-8de9-e0cd8f3a09ee None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:41:58.614837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a406c905-9248-4695-8de9-e0cd8f3a09ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:58.719088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406c905-9248-4695-8de9-e0cd8f3a09ee None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:41:58.793042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61b7c0-6cef-4108-b106-a6a62216eb4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:41:58.793650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61b7c0-6cef-4108-b106-a6a62216eb4a service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:41:58.793940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61b7c0-6cef-4108-b106-a6a62216eb4a service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:41:58.794220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61b7c0-6cef-4108-b106-a6a62216eb4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:41:58.795851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9a61b7c0-6cef-4108-b106-a6a62216eb4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:41:58.823024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:41:58.904436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61b7c0-6cef-4108-b106-a6a62216eb4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:41:58.906353 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61b7c0-6cef-4108-b106-a6a62216eb4a service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:41:58.924496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 653/1334] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:00.881227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9fccd-be7b-4d27-b5a8-b875ecc5bc38 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:00.881735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9fccd-be7b-4d27-b5a8-b875ecc5bc38 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:00.881735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9fccd-be7b-4d27-b5a8-b875ecc5bc38 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:00.883051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 682/1335] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:00.888405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86597966-5657-4f1f-9787-cb96969e0c6a None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:00.888790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86597966-5657-4f1f-9787-cb96969e0c6a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:00.888970 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86597966-5657-4f1f-9787-cb96969e0c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:00.954208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-86597966-5657-4f1f-9787-cb96969e0c6a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:00.955432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-86597966-5657-4f1f-9787-cb96969e0c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:01.031961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:01.056171 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 654/1336] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:01.345684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9fccd-be7b-4d27-b5a8-b875ecc5bc38 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:01.347226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c0e9fccd-be7b-4d27-b5a8-b875ecc5bc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:01.455975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9fccd-be7b-4d27-b5a8-b875ecc5bc38 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:01.483545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:01.528170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b17bc-32ad-4601-9711-7074a702ce11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:01.528984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b17bc-32ad-4601-9711-7074a702ce11 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:01.529494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b17bc-32ad-4601-9711-7074a702ce11 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:01.530127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b17bc-32ad-4601-9711-7074a702ce11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:01.531775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-128b17bc-32ad-4601-9711-7074a702ce11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:01.643649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b17bc-32ad-4601-9711-7074a702ce11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:01.645144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b17bc-32ad-4601-9711-7074a702ce11 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:01.663668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 683/1337] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:42:04.194277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f799-9aa0-4ed1-a2ee-ba8104ab295c None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:04.194866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f799-9aa0-4ed1-a2ee-ba8104ab295c None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:04.194969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f799-9aa0-4ed1-a2ee-ba8104ab295c None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:04.196711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 655/1338] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:04.202562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4a22-52ea-44d4-a664-1339e92faceb None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:04.203066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4a22-52ea-44d4-a664-1339e92faceb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:04.203165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4a22-52ea-44d4-a664-1339e92faceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:04.272906 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-cebc4a22-52ea-44d4-a664-1339e92faceb None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:04.274193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cebc4a22-52ea-44d4-a664-1339e92faceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:04.283635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:04.374164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 684/1339] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:04.658383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f799-9aa0-4ed1-a2ee-ba8104ab295c None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:04.659966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8866f799-9aa0-4ed1-a2ee-ba8104ab295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:04.774263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f799-9aa0-4ed1-a2ee-ba8104ab295c None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:04.851743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31023e4-0cb2-465e-bfcd-d0ba38ba040d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:04.852383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31023e4-0cb2-465e-bfcd-d0ba38ba040d service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:04.852666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31023e4-0cb2-465e-bfcd-d0ba38ba040d service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:04.852938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31023e4-0cb2-465e-bfcd-d0ba38ba040d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:04.854781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e31023e4-0cb2-465e-bfcd-d0ba38ba040d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:04.889793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:04.961219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31023e4-0cb2-465e-bfcd-d0ba38ba040d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:04.962651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31023e4-0cb2-465e-bfcd-d0ba38ba040d service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:04.980711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 656/1340] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:09.217845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92171aa8-dde0-4573-a782-c603dc1a060e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:09.218941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92171aa8-dde0-4573-a782-c603dc1a060e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:09.218941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92171aa8-dde0-4573-a782-c603dc1a060e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:09.220325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 685/1341] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:09.225604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a29da8-aaab-419d-8098-c878c707aeb2 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:09.226230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a29da8-aaab-419d-8098-c878c707aeb2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:09.226230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a29da8-aaab-419d-8098-c878c707aeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:09.293871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e9a29da8-aaab-419d-8098-c878c707aeb2 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:09.295139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e9a29da8-aaab-419d-8098-c878c707aeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:09.391463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:09.402808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 657/1342] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:09.684520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92171aa8-dde0-4573-a782-c603dc1a060e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:09.686095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-92171aa8-dde0-4573-a782-c603dc1a060e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:09.790622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92171aa8-dde0-4573-a782-c603dc1a060e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:09.833302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:09.871846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e8443-b4b4-4477-bd73-29eabed5b626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:09.872392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e8443-b4b4-4477-bd73-29eabed5b626 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:09.872671 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e8443-b4b4-4477-bd73-29eabed5b626 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:09.872947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e8443-b4b4-4477-bd73-29eabed5b626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:09.874545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-284e8443-b4b4-4477-bd73-29eabed5b626 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:09.986060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e8443-b4b4-4477-bd73-29eabed5b626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:09.987578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e8443-b4b4-4477-bd73-29eabed5b626 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:10.007981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 686/1343] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:13.029809 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1604-467f-46d8-b479-7aa3c758145a None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:13.030358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1604-467f-46d8-b479-7aa3c758145a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:13.030358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1604-467f-46d8-b479-7aa3c758145a None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:13.031807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 658/1344] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:13.037309 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7e65b-7a71-4e6b-9207-2613bb5d7339 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:13.037722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7e65b-7a71-4e6b-9207-2613bb5d7339 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:13.038014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7e65b-7a71-4e6b-9207-2613bb5d7339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:13.104667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-def7e65b-7a71-4e6b-9207-2613bb5d7339 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:13.105902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-def7e65b-7a71-4e6b-9207-2613bb5d7339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:13.125026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:13.206288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 687/1345] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:13.493345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1604-467f-46d8-b479-7aa3c758145a None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:13.494864 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a84b1604-467f-46d8-b479-7aa3c758145a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:13.609599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1604-467f-46d8-b479-7aa3c758145a None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:13.691455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cd3ed-acd1-442e-bdd8-9c075dd5ccad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:13.691455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cd3ed-acd1-442e-bdd8-9c075dd5ccad service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:13.691455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cd3ed-acd1-442e-bdd8-9c075dd5ccad service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:13.691455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cd3ed-acd1-442e-bdd8-9c075dd5ccad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:13.693152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ba5cd3ed-acd1-442e-bdd8-9c075dd5ccad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:13.740961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:13.815542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cd3ed-acd1-442e-bdd8-9c075dd5ccad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:13.817042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cd3ed-acd1-442e-bdd8-9c075dd5ccad service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:13.835833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 659/1346] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:18.367996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f45a1-1b93-4ceb-8a6d-91f788322e9f None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:18.368529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f45a1-1b93-4ceb-8a6d-91f788322e9f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:18.369655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f45a1-1b93-4ceb-8a6d-91f788322e9f None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:18.370106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 688/1347] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:18.376313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255679b0-4231-40b3-8a53-57be3e1b3635 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:18.376859 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255679b0-4231-40b3-8a53-57be3e1b3635 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:18.377073 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255679b0-4231-40b3-8a53-57be3e1b3635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:18.453271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-255679b0-4231-40b3-8a53-57be3e1b3635 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:18.454780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-255679b0-4231-40b3-8a53-57be3e1b3635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:18.563261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:18.566059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 660/1348] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:18.863794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f45a1-1b93-4ceb-8a6d-91f788322e9f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:18.865370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-af4f45a1-1b93-4ceb-8a6d-91f788322e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:18.977969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f45a1-1b93-4ceb-8a6d-91f788322e9f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:19.026436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:19.054422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5433af-08d7-4511-81b3-a76f1ddae486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:19.055037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5433af-08d7-4511-81b3-a76f1ddae486 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:19.055324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5433af-08d7-4511-81b3-a76f1ddae486 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:19.055599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5433af-08d7-4511-81b3-a76f1ddae486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:19.057248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6c5433af-08d7-4511-81b3-a76f1ddae486 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:19.164335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5433af-08d7-4511-81b3-a76f1ddae486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:19.165782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5433af-08d7-4511-81b3-a76f1ddae486 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:19.185243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 689/1349] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:24.171614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d67d2b-75bd-4b29-8360-87101cb3887e None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:24.172136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d67d2b-75bd-4b29-8360-87101cb3887e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:24.172353 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d67d2b-75bd-4b29-8360-87101cb3887e None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:24.174645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 661/1350] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 15:42:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:24.179974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c7f3d-9148-4f2f-a77f-a38cce3526ef None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:24.180399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c7f3d-9148-4f2f-a77f-a38cce3526ef None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:24.184010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c7f3d-9148-4f2f-a77f-a38cce3526ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:24.325564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-5c1c7f3d-9148-4f2f-a77f-a38cce3526ef None None] MFA Rules not processed for user `b28f987c90df4c19a12bb1819dc65cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:24.327310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5c1c7f3d-9148-4f2f-a77f-a38cce3526ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:24.347516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:24.447680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 690/1351] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 15:42:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:28.688449 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b5b1-b7cc-4a6b-9fe7-35f966077302 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:28.688979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b5b1-b7cc-4a6b-9fe7-35f966077302 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:28.689059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b5b1-b7cc-4a6b-9fe7-35f966077302 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:28.690586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 662/1352] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 15:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:28.696044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115aadad-ebc7-4c4d-ab10-c5009d99e6ae None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:28.696557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115aadad-ebc7-4c4d-ab10-c5009d99e6ae None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:28.696923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115aadad-ebc7-4c4d-ab10-c5009d99e6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:28.776954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-115aadad-ebc7-4c4d-ab10-c5009d99e6ae None None] MFA Rules not processed for user `b28f987c90df4c19a12bb1819dc65cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:28.778174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-115aadad-ebc7-4c4d-ab10-c5009d99e6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:28.852581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:28.889023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 691/1353] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 15:42:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:30.166277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab68f5-747a-431a-a567-077283c85b95 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:30.166795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab68f5-747a-431a-a567-077283c85b95 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:30.167186 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab68f5-747a-431a-a567-077283c85b95 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:30.169123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 663/1354] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 15:42:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:30.177091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec203f1-5548-41d2-8333-ebc97570b329 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:30.177091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec203f1-5548-41d2-8333-ebc97570b329 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:30.177091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec203f1-5548-41d2-8333-ebc97570b329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:30.372156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-4ec203f1-5548-41d2-8333-ebc97570b329 None None] MFA Rules not processed for user `b28f987c90df4c19a12bb1819dc65cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:30.373354 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4ec203f1-5548-41d2-8333-ebc97570b329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:30.486958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 692/1355] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 15:42:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:33.823900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf05ab5-b9ef-45a1-8adc-996d788c43e9 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:33.824554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf05ab5-b9ef-45a1-8adc-996d788c43e9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:33.824554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf05ab5-b9ef-45a1-8adc-996d788c43e9 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:33.825960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 664/1356] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:33.831507 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa79ff-a505-41a9-8492-b21c6e54a290 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:33.832204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa79ff-a505-41a9-8492-b21c6e54a290 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:33.832204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa79ff-a505-41a9-8492-b21c6e54a290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:33.868042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:33.899998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-26aa79ff-a505-41a9-8492-b21c6e54a290 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:33.901390 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-26aa79ff-a505-41a9-8492-b21c6e54a290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:34.054315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 693/1357] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:34.363318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef965-563f-4aa7-a042-8274bd59f8f3 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:34.364444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef965-563f-4aa7-a042-8274bd59f8f3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:34.364985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef965-563f-4aa7-a042-8274bd59f8f3 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:34.367136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 665/1358] 10.222.0.98 () {54 vars in 962 bytes} [Fri Aug 26 15:42:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:34.377949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd89d2a-f7ed-4d97-8dbf-dd9852a70c51 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:34.378527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd89d2a-f7ed-4d97-8dbf-dd9852a70c51 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:34.379136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd89d2a-f7ed-4d97-8dbf-dd9852a70c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:34.470419 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-abd89d2a-f7ed-4d97-8dbf-dd9852a70c51 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:34.471652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-abd89d2a-f7ed-4d97-8dbf-dd9852a70c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:34.540258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:34.624740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 694/1359] 10.222.0.98 () {58 vars in 1059 bytes} [Fri Aug 26 15:42:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:34.626319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef965-563f-4aa7-a042-8274bd59f8f3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:34.628333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-101ef965-563f-4aa7-a042-8274bd59f8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:34.799692 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef965-563f-4aa7-a042-8274bd59f8f3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:34.881687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:34.927793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3fa25-dcd2-42cd-ae77-3d8dda307d5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:34.928758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3fa25-dcd2-42cd-ae77-3d8dda307d5e service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:34.929276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3fa25-dcd2-42cd-ae77-3d8dda307d5e service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:34.929725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3fa25-dcd2-42cd-ae77-3d8dda307d5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:34.931613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d8b3fa25-dcd2-42cd-ae77-3d8dda307d5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:35.161942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3fa25-dcd2-42cd-ae77-3d8dda307d5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:35.163661 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3fa25-dcd2-42cd-ae77-3d8dda307d5e service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:35.185110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 666/1360] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:37.400979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d4049-33e7-49b6-8ebe-c2a943d37e4c None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:37.401571 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d4049-33e7-49b6-8ebe-c2a943d37e4c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:37.401571 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d4049-33e7-49b6-8ebe-c2a943d37e4c None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:37.403024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 695/1361] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:37.408714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602f166-8b90-4a90-a147-8c979dcb79eb None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:37.409395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602f166-8b90-4a90-a147-8c979dcb79eb None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:37.409781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602f166-8b90-4a90-a147-8c979dcb79eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:37.510109 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-4602f166-8b90-4a90-a147-8c979dcb79eb None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:37.511437 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4602f166-8b90-4a90-a147-8c979dcb79eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:37.554476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:37.633781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 667/1362] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:37.922266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d4049-33e7-49b6-8ebe-c2a943d37e4c None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:37.923790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6d2d4049-33e7-49b6-8ebe-c2a943d37e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:38.092813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d4049-33e7-49b6-8ebe-c2a943d37e4c None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:38.103175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:38.251202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db28f73-110a-4c2b-ade6-615fa7105192 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:38.252437 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db28f73-110a-4c2b-ade6-615fa7105192 service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:38.252865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db28f73-110a-4c2b-ade6-615fa7105192 service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:38.253280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db28f73-110a-4c2b-ade6-615fa7105192 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:38.255981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3db28f73-110a-4c2b-ade6-615fa7105192 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:38.380306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db28f73-110a-4c2b-ade6-615fa7105192 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:38.382039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db28f73-110a-4c2b-ade6-615fa7105192 service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:38.404371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 696/1363] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:40.832998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2822dda-cbed-4622-bfef-718340073951 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:40.833586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2822dda-cbed-4622-bfef-718340073951 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:40.833586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2822dda-cbed-4622-bfef-718340073951 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:40.835040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 668/1364] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:40.840188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f6d71-b142-4088-8a1f-ed7f42fc4cdd None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:40.840601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f6d71-b142-4088-8a1f-ed7f42fc4cdd None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:40.840776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f6d71-b142-4088-8a1f-ed7f42fc4cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:40.857414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:40.912130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-b36f6d71-b142-4088-8a1f-ed7f42fc4cdd None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:40.913544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-b36f6d71-b142-4088-8a1f-ed7f42fc4cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:41.015919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 697/1365] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:41.316075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c24d4b-fb8c-4267-abaf-16d6c297e840 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:41.316402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c24d4b-fb8c-4267-abaf-16d6c297e840 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:41.316668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c24d4b-fb8c-4267-abaf-16d6c297e840 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:41.318221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 669/1366] 10.222.0.98 () {54 vars in 962 bytes} [Fri Aug 26 15:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:41.323515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40048e1c-a7f2-4ca2-9d36-379a4e2a06c6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:41.323961 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40048e1c-a7f2-4ca2-9d36-379a4e2a06c6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:41.324358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40048e1c-a7f2-4ca2-9d36-379a4e2a06c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:41.406352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-40048e1c-a7f2-4ca2-9d36-379a4e2a06c6 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:41.407622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-40048e1c-a7f2-4ca2-9d36-379a4e2a06c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:41.446479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:41.539662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 698/1367] 10.222.0.98 () {58 vars in 1059 bytes} [Fri Aug 26 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:41.548546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c24d4b-fb8c-4267-abaf-16d6c297e840 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:41.550377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-99c24d4b-fb8c-4267-abaf-16d6c297e840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:41.712874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c24d4b-fb8c-4267-abaf-16d6c297e840 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:41.736462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:41.840016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae26595-f4c4-4e46-a0b7-57df3a1bcacc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:41.840638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae26595-f4c4-4e46-a0b7-57df3a1bcacc service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:41.840947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae26595-f4c4-4e46-a0b7-57df3a1bcacc service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:41.841129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae26595-f4c4-4e46-a0b7-57df3a1bcacc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:41.842899 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6ae26595-f4c4-4e46-a0b7-57df3a1bcacc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:41.940840 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae26595-f4c4-4e46-a0b7-57df3a1bcacc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:41.942452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae26595-f4c4-4e46-a0b7-57df3a1bcacc service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:41.962546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 670/1368] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:42.121878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e33b9-aed2-468a-87a8-c69c7e4ed2b5 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:42.122964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e33b9-aed2-468a-87a8-c69c7e4ed2b5 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:42.122964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e33b9-aed2-468a-87a8-c69c7e4ed2b5 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:42.128643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 699/1369] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:42.131090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa5099-6af5-4106-a76d-e4e3cb658d06 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:42.131521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa5099-6af5-4106-a76d-e4e3cb658d06 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:42.131909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa5099-6af5-4106-a76d-e4e3cb658d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:42.165234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:42.201633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-06fa5099-6af5-4106-a76d-e4e3cb658d06 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:42.202934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-06fa5099-6af5-4106-a76d-e4e3cb658d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:42.328973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 671/1370] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:42.914801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5bbf8-9755-4650-bd21-6b34e17d237d None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:42.914801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5bbf8-9755-4650-bd21-6b34e17d237d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:42.914801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5bbf8-9755-4650-bd21-6b34e17d237d None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:42.917273 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 700/1371] 10.222.0.98 () {54 vars in 964 bytes} [Fri Aug 26 15:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:42.921069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0557f-e605-414c-b691-c40779d3e8de None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:42.921665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0557f-e605-414c-b691-c40779d3e8de None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:42.921665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0557f-e605-414c-b691-c40779d3e8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:43.003156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-00f0557f-e605-414c-b691-c40779d3e8de None None] MFA Rules not processed for user `3a1ff8aff9e048c6972843e0b50c4d45`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:43.004411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-00f0557f-e605-414c-b691-c40779d3e8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:43.051758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:43.095769 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 672/1372] 10.222.0.98 () {58 vars in 1061 bytes} [Fri Aug 26 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:43.103365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5bbf8-9755-4650-bd21-6b34e17d237d None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:43.104900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-27a5bbf8-9755-4650-bd21-6b34e17d237d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:43.225470 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5bbf8-9755-4650-bd21-6b34e17d237d None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:43.269897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:43.308185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d99cc-a9bf-4b95-bcfe-13938dec0591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:43.308803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d99cc-a9bf-4b95-bcfe-13938dec0591 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:43.309089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d99cc-a9bf-4b95-bcfe-13938dec0591 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:43.309370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d99cc-a9bf-4b95-bcfe-13938dec0591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:43.311046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ec6d99cc-a9bf-4b95-bcfe-13938dec0591 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:43.436703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d99cc-a9bf-4b95-bcfe-13938dec0591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:43.438164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d99cc-a9bf-4b95-bcfe-13938dec0591 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:43.458388 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 701/1373] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:44.221142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e33306-7df5-4cad-aee3-285f4843c246 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:44.226095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e33306-7df5-4cad-aee3-285f4843c246 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:44.226095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e33306-7df5-4cad-aee3-285f4843c246 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:44.226095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 673/1374] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:44.228748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c34edb-2c30-4b0c-80e2-d193a1fbaaae None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:44.229160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c34edb-2c30-4b0c-80e2-d193a1fbaaae None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:44.229320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c34edb-2c30-4b0c-80e2-d193a1fbaaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:44.297945 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-82c34edb-2c30-4b0c-80e2-d193a1fbaaae None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:44.299414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-82c34edb-2c30-4b0c-80e2-d193a1fbaaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:44.320632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:44.398143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 702/1375] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:42:44.695027 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e33306-7df5-4cad-aee3-285f4843c246 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:44.696701 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-01e33306-7df5-4cad-aee3-285f4843c246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:44.845478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e33306-7df5-4cad-aee3-285f4843c246 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:44.901142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:44.964192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8d932-1cd4-4c5e-886c-e9aaf717851d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:44.964737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8d932-1cd4-4c5e-886c-e9aaf717851d service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:44.965017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8d932-1cd4-4c5e-886c-e9aaf717851d service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:44.965289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8d932-1cd4-4c5e-886c-e9aaf717851d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:44.967458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c7c8d932-1cd4-4c5e-886c-e9aaf717851d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:45.061411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8d932-1cd4-4c5e-886c-e9aaf717851d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:45.063038 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8d932-1cd4-4c5e-886c-e9aaf717851d service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:45.084175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 674/1376] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:47.167335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6faa-a2b8-4226-b9d6-556d6027d808 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:47.167900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6faa-a2b8-4226-b9d6-556d6027d808 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:47.167900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6faa-a2b8-4226-b9d6-556d6027d808 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:47.169558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 703/1377] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:47.175257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a90d9b-4482-4ea1-b4e4-98f413d73261 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:47.175759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a90d9b-4482-4ea1-b4e4-98f413d73261 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:47.175844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a90d9b-4482-4ea1-b4e4-98f413d73261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:47.265400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-41a90d9b-4482-4ea1-b4e4-98f413d73261 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:47.266621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-41a90d9b-4482-4ea1-b4e4-98f413d73261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:47.274259 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:47.404027 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 675/1378] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:47.701064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf6faa-a2b8-4226-b9d6-556d6027d808 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:47.702604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1ccf6faa-a2b8-4226-b9d6-556d6027d808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:47.855680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf6faa-a2b8-4226-b9d6-556d6027d808 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:47.860254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:47.981990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dc3aa-8334-4902-9c26-554d0856f0d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:47.982492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc3aa-8334-4902-9c26-554d0856f0d9 service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:47.982780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc3aa-8334-4902-9c26-554d0856f0d9 service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:47.983057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc3aa-8334-4902-9c26-554d0856f0d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:47.984683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-300dc3aa-8334-4902-9c26-554d0856f0d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:48.079202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dc3aa-8334-4902-9c26-554d0856f0d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:48.080821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dc3aa-8334-4902-9c26-554d0856f0d9 service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:48.100462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 704/1379] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:50.219041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b4f01-5467-4b8e-8598-4e9bd95c3701 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:50.219576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b4f01-5467-4b8e-8598-4e9bd95c3701 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:50.219830 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b4f01-5467-4b8e-8598-4e9bd95c3701 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:50.221499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 676/1380] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:50.227119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce223-26f4-4a58-a33f-3b6cb626a392 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:50.227607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce223-26f4-4a58-a33f-3b6cb626a392 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:50.227705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce223-26f4-4a58-a33f-3b6cb626a392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:50.240367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:50.299264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-b8dce223-26f4-4a58-a33f-3b6cb626a392 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:50.301209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-b8dce223-26f4-4a58-a33f-3b6cb626a392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:50.412568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 705/1381] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:50.704008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b4f01-5467-4b8e-8598-4e9bd95c3701 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:50.705722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b31b4f01-5467-4b8e-8598-4e9bd95c3701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:50.868852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:50.878925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b4f01-5467-4b8e-8598-4e9bd95c3701 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:50.991739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde33e29-ee86-4394-b0ec-7c6cdf908163 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:50.992579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde33e29-ee86-4394-b0ec-7c6cdf908163 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:50.993029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde33e29-ee86-4394-b0ec-7c6cdf908163 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:50.993477 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde33e29-ee86-4394-b0ec-7c6cdf908163 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:50.995063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-bde33e29-ee86-4394-b0ec-7c6cdf908163 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:51.101582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde33e29-ee86-4394-b0ec-7c6cdf908163 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:51.103476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde33e29-ee86-4394-b0ec-7c6cdf908163 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:51.127594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:51.155641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 677/1382] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:42:53.289908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8630a-e972-4fab-95f4-c04547a8be26 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:53.290448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8630a-e972-4fab-95f4-c04547a8be26 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:53.290542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8630a-e972-4fab-95f4-c04547a8be26 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:53.291853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 706/1383] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:53.297994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef506e2-4b78-487e-b892-0324317d3f6d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:53.298433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef506e2-4b78-487e-b892-0324317d3f6d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:53.299049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef506e2-4b78-487e-b892-0324317d3f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:53.378793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-4ef506e2-4b78-487e-b892-0324317d3f6d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:53.380345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4ef506e2-4b78-487e-b892-0324317d3f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:53.489264 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 678/1384] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:53.788326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8630a-e972-4fab-95f4-c04547a8be26 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:53.789844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-d4d8630a-e972-4fab-95f4-c04547a8be26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:53.879313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:53.945886 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8630a-e972-4fab-95f4-c04547a8be26 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:54.093252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e680e1f-72a1-4c0b-9928-d4dd512ef91f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:54.094134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e680e1f-72a1-4c0b-9928-d4dd512ef91f service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:54.094549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e680e1f-72a1-4c0b-9928-d4dd512ef91f service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:54.095003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e680e1f-72a1-4c0b-9928-d4dd512ef91f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:54.097580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-7e680e1f-72a1-4c0b-9928-d4dd512ef91f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:54.163963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:54.209121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e680e1f-72a1-4c0b-9928-d4dd512ef91f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:54.211944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e680e1f-72a1-4c0b-9928-d4dd512ef91f service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:54.236502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 707/1385] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:56.188352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495577a5-01b4-42d0-a366-7728253fa976 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:56.188918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495577a5-01b4-42d0-a366-7728253fa976 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:56.188918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495577a5-01b4-42d0-a366-7728253fa976 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:56.190784 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 679/1386] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:56.196285 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb41f5-8201-46de-8943-ba9bae2a13f8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:56.196863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb41f5-8201-46de-8943-ba9bae2a13f8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:56.196863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb41f5-8201-46de-8943-ba9bae2a13f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:56.270015 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-4fbb41f5-8201-46de-8943-ba9bae2a13f8 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:56.271276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4fbb41f5-8201-46de-8943-ba9bae2a13f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:56.384327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 708/1387] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:56.675161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495577a5-01b4-42d0-a366-7728253fa976 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:56.676717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-495577a5-01b4-42d0-a366-7728253fa976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:56.757605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:56.846996 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495577a5-01b4-42d0-a366-7728253fa976 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:56.987258 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8fb1f-c2da-4136-a262-0ecdc14fa91a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:56.988257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8fb1f-c2da-4136-a262-0ecdc14fa91a service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:56.988616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8fb1f-c2da-4136-a262-0ecdc14fa91a service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:56.988971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8fb1f-c2da-4136-a262-0ecdc14fa91a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:56.991121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b3d8fb1f-c2da-4136-a262-0ecdc14fa91a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:57.041867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:57.103850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8fb1f-c2da-4136-a262-0ecdc14fa91a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:42:57.105496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8fb1f-c2da-4136-a262-0ecdc14fa91a service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:42:57.126771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 680/1388] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:42:59.235661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3611e-8193-4624-8ffc-6cac05b2d776 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:59.236259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3611e-8193-4624-8ffc-6cac05b2d776 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:59.236259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3611e-8193-4624-8ffc-6cac05b2d776 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:59.237737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 709/1389] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:59.243067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f54fd7-5dc3-4749-80c2-074911439458 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:59.243506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f54fd7-5dc3-4749-80c2-074911439458 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:59.243632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f54fd7-5dc3-4749-80c2-074911439458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:59.294563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdb1e7-bd53-4327-839c-8cb1ab82deeb None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:59.294853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdb1e7-bd53-4327-839c-8cb1ab82deeb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:59.295138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdb1e7-bd53-4327-839c-8cb1ab82deeb None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:59.296508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 710/1390] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:42:59.302256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0bc53-76c1-42b9-a43b-1d6f198d8411 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:59.302546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0bc53-76c1-42b9-a43b-1d6f198d8411 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:59.302829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0bc53-76c1-42b9-a43b-1d6f198d8411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:59.313727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-84f54fd7-5dc3-4749-80c2-074911439458 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:59.314979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-84f54fd7-5dc3-4749-80c2-074911439458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:59.340500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:59.371882 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-32a0bc53-76c1-42b9-a43b-1d6f198d8411 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:59.373780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-32a0bc53-76c1-42b9-a43b-1d6f198d8411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:59.428052 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 681/1391] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:42:59.475034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 711/1392] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:42:59.720134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f54fd7-5dc3-4749-80c2-074911439458 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:59.721373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-84f54fd7-5dc3-4749-80c2-074911439458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:59.735904 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:59.775351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be73072-f76c-470e-a3b4-863f8c54d9c9 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:59.775648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be73072-f76c-470e-a3b4-863f8c54d9c9 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:59.775966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be73072-f76c-470e-a3b4-863f8c54d9c9 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:59.777912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 712/1393] 10.222.0.98 () {54 vars in 967 bytes} [Fri Aug 26 15:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:42:59.782750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b1565-993d-43f7-91af-e5c7d4a53929 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:59.783157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b1565-993d-43f7-91af-e5c7d4a53929 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:59.783444 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b1565-993d-43f7-91af-e5c7d4a53929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:59.839698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f54fd7-5dc3-4749-80c2-074911439458 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:59.859616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f59b1565-993d-43f7-91af-e5c7d4a53929 None None] MFA Rules not processed for user `cef193a54eff44f2b53716e3ec1a29cf`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:42:59.860791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f59b1565-993d-43f7-91af-e5c7d4a53929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:59.909052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:42:59.940177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 713/1394] 10.222.0.98 () {58 vars in 1064 bytes} [Fri Aug 26 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:42:59.947532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b1565-993d-43f7-91af-e5c7d4a53929 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:42:59.955821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b1565-993d-43f7-91af-e5c7d4a53929 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:42:59.957184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2baeb-84a5-44fa-8bdb-30dee0ac1846 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:42:59.957790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2baeb-84a5-44fa-8bdb-30dee0ac1846 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:42:59.958067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2baeb-84a5-44fa-8bdb-30dee0ac1846 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:42:59.958464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2baeb-84a5-44fa-8bdb-30dee0ac1846 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:42:59.960075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7af2baeb-84a5-44fa-8bdb-30dee0ac1846 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:42:59.968771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:00.030485 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5510bca-dfb4-4a48-ac53-4468004a3296 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cef193a54eff44f2b53716e3ec1a29cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:00.031096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5510bca-dfb4-4a48-ac53-4468004a3296 service glance] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:00.031378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5510bca-dfb4-4a48-ac53-4468004a3296 service glance] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:00.031655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5510bca-dfb4-4a48-ac53-4468004a3296 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:00.033292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f5510bca-dfb4-4a48-ac53-4468004a3296 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:00.051057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2baeb-84a5-44fa-8bdb-30dee0ac1846 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:00.052683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2baeb-84a5-44fa-8bdb-30dee0ac1846 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:00.070623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 682/1395] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:00.144126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5510bca-dfb4-4a48-ac53-4468004a3296 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:00.145626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5510bca-dfb4-4a48-ac53-4468004a3296 service glance] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:00.159808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:00.164803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 714/1396] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:43:01.050648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacac97-661a-481a-b646-ee01dc9a04af None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:01.051126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacac97-661a-481a-b646-ee01dc9a04af None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:01.051230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacac97-661a-481a-b646-ee01dc9a04af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:01.120291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-4cacac97-661a-481a-b646-ee01dc9a04af None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:01.122365 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4cacac97-661a-481a-b646-ee01dc9a04af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:01.210246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:01.213481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 683/1397] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:01.261120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316696b-65e5-4ca5-87f0-2ab8324b4235 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:01.261610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316696b-65e5-4ca5-87f0-2ab8324b4235 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:01.261903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316696b-65e5-4ca5-87f0-2ab8324b4235 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:01.263335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 715/1398] 10.222.0.98 () {54 vars in 943 bytes} [Fri Aug 26 15:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:01.269827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872b309-6d30-47c3-8a1c-4864e07dc531 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:01.270116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872b309-6d30-47c3-8a1c-4864e07dc531 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:01.270395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872b309-6d30-47c3-8a1c-4864e07dc531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:01.381623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-4872b309-6d30-47c3-8a1c-4864e07dc531 None None] MFA Rules not processed for user `1fae47d6b7a7472a9d94636bb126f802`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:01.383967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4872b309-6d30-47c3-8a1c-4864e07dc531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:01.514914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 684/1399] 10.222.0.98 () {58 vars in 1040 bytes} [Fri Aug 26 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:01.526718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316696b-65e5-4ca5-87f0-2ab8324b4235 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:01.528453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a316696b-65e5-4ca5-87f0-2ab8324b4235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:01.679652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316696b-65e5-4ca5-87f0-2ab8324b4235 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:01.765229 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:01.783912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd7132-067f-4ea7-bf35-ce5da5ae4fab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:01.784535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7132-067f-4ea7-bf35-ce5da5ae4fab service swift] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:01.784816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7132-067f-4ea7-bf35-ce5da5ae4fab service swift] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:01.785095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7132-067f-4ea7-bf35-ce5da5ae4fab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:01.786733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-efbd7132-067f-4ea7-bf35-ce5da5ae4fab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:01.872447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd7132-067f-4ea7-bf35-ce5da5ae4fab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:01.874062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd7132-067f-4ea7-bf35-ce5da5ae4fab service swift] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:01.881975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 716/1400] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:43:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:43:02.206346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbe39f-8661-461c-a079-19911c26e31a None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:02.206799 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbe39f-8661-461c-a079-19911c26e31a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:02.206891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbe39f-8661-461c-a079-19911c26e31a None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:02.208642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 685/1401] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:02.213878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859311b-b89c-493d-81e7-da151af80f8b None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:02.214166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859311b-b89c-493d-81e7-da151af80f8b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:02.214448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859311b-b89c-493d-81e7-da151af80f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:02.282622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-5859311b-b89c-493d-81e7-da151af80f8b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:02.283817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5859311b-b89c-493d-81e7-da151af80f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:02.356314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:02.398015 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 717/1402] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:02.697757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbe39f-8661-461c-a079-19911c26e31a None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:02.699987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cbdbe39f-8661-461c-a079-19911c26e31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:02.767207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:02.865531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbe39f-8661-461c-a079-19911c26e31a None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:03.002620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c95d7-b02d-43da-bab4-e1c7fc1e845f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:03.003539 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c95d7-b02d-43da-bab4-e1c7fc1e845f service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:03.004008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c95d7-b02d-43da-bab4-e1c7fc1e845f service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:03.004428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c95d7-b02d-43da-bab4-e1c7fc1e845f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:03.008339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-686c95d7-b02d-43da-bab4-e1c7fc1e845f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:03.030698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:03.111922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c95d7-b02d-43da-bab4-e1c7fc1e845f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:03.113691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c95d7-b02d-43da-bab4-e1c7fc1e845f service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:03.135585 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 686/1403] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:05.313492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cac40-4de2-4b50-b63c-8149e0ce39ea None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:05.314433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cac40-4de2-4b50-b63c-8149e0ce39ea None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:05.314717 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cac40-4de2-4b50-b63c-8149e0ce39ea None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:05.316420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 718/1404] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:05.321439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b474ba-96e3-44e2-bbd1-c96a127cb14e None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:05.321890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b474ba-96e3-44e2-bbd1-c96a127cb14e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:05.321990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b474ba-96e3-44e2-bbd1-c96a127cb14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:05.397108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-a8b474ba-96e3-44e2-bbd1-c96a127cb14e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:05.397967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a8b474ba-96e3-44e2-bbd1-c96a127cb14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:05.500822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:05.507222 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 687/1405] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:05.800235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cac40-4de2-4b50-b63c-8149e0ce39ea None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:05.801839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f13cac40-4de2-4b50-b63c-8149e0ce39ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:05.970583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cac40-4de2-4b50-b63c-8149e0ce39ea None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:06.033212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:06.103390 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3447b-3c6d-4ad6-9239-0ddeafa747a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:06.104242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3447b-3c6d-4ad6-9239-0ddeafa747a7 service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:06.104535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3447b-3c6d-4ad6-9239-0ddeafa747a7 service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:06.104816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3447b-3c6d-4ad6-9239-0ddeafa747a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:06.106475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-edc3447b-3c6d-4ad6-9239-0ddeafa747a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:06.206344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3447b-3c6d-4ad6-9239-0ddeafa747a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:06.208022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3447b-3c6d-4ad6-9239-0ddeafa747a7 service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:06.229579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 719/1406] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:08.460999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c5f84-a983-4269-9011-fd6b59749d65 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:08.460999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c5f84-a983-4269-9011-fd6b59749d65 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:08.461648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c5f84-a983-4269-9011-fd6b59749d65 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:08.462440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 688/1407] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:08.468537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a9e07-e2ed-4d69-bebb-957e843180f5 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:08.468973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a9e07-e2ed-4d69-bebb-957e843180f5 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:08.469149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a9e07-e2ed-4d69-bebb-957e843180f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:08.546070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9a1a9e07-e2ed-4d69-bebb-957e843180f5 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:08.547299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9a1a9e07-e2ed-4d69-bebb-957e843180f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:08.561578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:08.663883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 720/1408] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:08.966294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c5f84-a983-4269-9011-fd6b59749d65 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:08.967888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f93c5f84-a983-4269-9011-fd6b59749d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:09.155573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c5f84-a983-4269-9011-fd6b59749d65 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:09.242903 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:09.300855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ff226-f2a6-4350-88e8-e5c4b9e7dd39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:09.300855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ff226-f2a6-4350-88e8-e5c4b9e7dd39 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:09.302974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ff226-f2a6-4350-88e8-e5c4b9e7dd39 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:09.302974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ff226-f2a6-4350-88e8-e5c4b9e7dd39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:09.304477 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-532ff226-f2a6-4350-88e8-e5c4b9e7dd39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:09.418090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ff226-f2a6-4350-88e8-e5c4b9e7dd39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:09.419740 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ff226-f2a6-4350-88e8-e5c4b9e7dd39 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:09.440224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 689/1409] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:18.661916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=38 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:19.698418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=64 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:25.530134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6a96-1bfe-4d9e-ba6f-8d13b194357c None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:25.530681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6a96-1bfe-4d9e-ba6f-8d13b194357c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:25.530681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6a96-1bfe-4d9e-ba6f-8d13b194357c None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:25.532067 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 721/1410] 10.222.0.98 () {54 vars in 918 bytes} [Fri Aug 26 15:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:25.537177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993804b-07d7-4d35-a0e3-49e07862c2bd None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:25.537591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993804b-07d7-4d35-a0e3-49e07862c2bd None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:25.537742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993804b-07d7-4d35-a0e3-49e07862c2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:25.617042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b993804b-07d7-4d35-a0e3-49e07862c2bd None None] MFA Rules not processed for user `b28f987c90df4c19a12bb1819dc65cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:25.618275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b993804b-07d7-4d35-a0e3-49e07862c2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:25.743612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 690/1411] 10.222.0.98 () {58 vars in 1015 bytes} [Fri Aug 26 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:25.758075 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f436272-09be-4cf0-961a-fd10974b7aed None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:25.758369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f436272-09be-4cf0-961a-fd10974b7aed None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:25.758653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f436272-09be-4cf0-961a-fd10974b7aed None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:25.759998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 722/1412] 10.222.0.98 () {54 vars in 918 bytes} [Fri Aug 26 15:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:25.764552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c98ad-b470-4790-b555-519f8675a3ba None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:25.764843 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c98ad-b470-4790-b555-519f8675a3ba None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:25.765124 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c98ad-b470-4790-b555-519f8675a3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:25.839086 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:25.841471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-922c98ad-b470-4790-b555-519f8675a3ba None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:25.842695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-922c98ad-b470-4790-b555-519f8675a3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:25.957267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 691/1413] 10.222.0.98 () {58 vars in 1015 bytes} [Fri Aug 26 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:28.371456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9063-6b2b-44f1-bf27-76f16db2ad19 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:28.372089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9063-6b2b-44f1-bf27-76f16db2ad19 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:28.372089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9063-6b2b-44f1-bf27-76f16db2ad19 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:28.373314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 723/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:28.378554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f0e90-e1d0-415f-8693-10d207f81edc None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:28.378978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f0e90-e1d0-415f-8693-10d207f81edc None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:28.379115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f0e90-e1d0-415f-8693-10d207f81edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:28.445324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-019f0e90-e1d0-415f-8693-10d207f81edc None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:28.446538 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-019f0e90-e1d0-415f-8693-10d207f81edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:28.501938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:28.543898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 692/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:28.832816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e9063-6b2b-44f1-bf27-76f16db2ad19 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:28.834339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9f7e9063-6b2b-44f1-bf27-76f16db2ad19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:28.980001 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e9063-6b2b-44f1-bf27-76f16db2ad19 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:29.051986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:29.092856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4fbd2-0c7c-4380-be91-344f31bea9f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:29.093382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4fbd2-0c7c-4380-be91-344f31bea9f8 service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:29.093663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4fbd2-0c7c-4380-be91-344f31bea9f8 service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:29.093940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4fbd2-0c7c-4380-be91-344f31bea9f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:29.095537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c2b4fbd2-0c7c-4380-be91-344f31bea9f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:29.183932 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4fbd2-0c7c-4380-be91-344f31bea9f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:29.185580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4fbd2-0c7c-4380-be91-344f31bea9f8 service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:29.203824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 724/1416] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:30.097923 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df29a8-fcff-410f-b584-59814b26fd55 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:30.098491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df29a8-fcff-410f-b584-59814b26fd55 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:30.098491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df29a8-fcff-410f-b584-59814b26fd55 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:30.099817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 693/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:30.105474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9f18b-2638-44a4-b97a-0519560c30f8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:30.105888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9f18b-2638-44a4-b97a-0519560c30f8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:30.106037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9f18b-2638-44a4-b97a-0519560c30f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:30.184998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-30e9f18b-2638-44a4-b97a-0519560c30f8 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:30.186216 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-30e9f18b-2638-44a4-b97a-0519560c30f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:30.212933 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:30.282575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 725/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:30.446712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df29a8-fcff-410f-b584-59814b26fd55 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:30.448343 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-20df29a8-fcff-410f-b584-59814b26fd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:30.597611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df29a8-fcff-410f-b584-59814b26fd55 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:30.639694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:30.714148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92007fa-4c6c-4570-8e8d-c77e2059e1f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:30.714681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92007fa-4c6c-4570-8e8d-c77e2059e1f7 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:30.714967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92007fa-4c6c-4570-8e8d-c77e2059e1f7 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:30.715241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92007fa-4c6c-4570-8e8d-c77e2059e1f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:30.716905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e92007fa-4c6c-4570-8e8d-c77e2059e1f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:30.812090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92007fa-4c6c-4570-8e8d-c77e2059e1f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:30.813693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92007fa-4c6c-4570-8e8d-c77e2059e1f7 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:30.831965 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 694/1419] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:31.739593 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024c30c-fa46-4df2-979c-4495392a7de6 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:31.740179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024c30c-fa46-4df2-979c-4495392a7de6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:31.740179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024c30c-fa46-4df2-979c-4495392a7de6 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:31.741612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 726/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:31.747197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bad3c-ad5f-4843-ace8-e6c5c000bdca None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:31.747619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bad3c-ad5f-4843-ace8-e6c5c000bdca None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:31.747757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bad3c-ad5f-4843-ace8-e6c5c000bdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:31.811550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:31.831009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-6e7bad3c-ad5f-4843-ace8-e6c5c000bdca None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:31.832268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6e7bad3c-ad5f-4843-ace8-e6c5c000bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:31.926553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 695/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:32.088744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024c30c-fa46-4df2-979c-4495392a7de6 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:32.090274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2024c30c-fa46-4df2-979c-4495392a7de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:32.235954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024c30c-fa46-4df2-979c-4495392a7de6 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:32.248655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:32.349451 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794917f-e4f5-4a39-b6a6-ed6142e74f2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:32.350057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794917f-e4f5-4a39-b6a6-ed6142e74f2a service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:32.350342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794917f-e4f5-4a39-b6a6-ed6142e74f2a service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:32.350620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794917f-e4f5-4a39-b6a6-ed6142e74f2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:32.352298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-7794917f-e4f5-4a39-b6a6-ed6142e74f2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:32.441754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794917f-e4f5-4a39-b6a6-ed6142e74f2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:32.443355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794917f-e4f5-4a39-b6a6-ed6142e74f2a service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:32.461345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 727/1422] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:33.330072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba190c68-0a3d-4a87-9359-e3ae761327f4 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:33.330646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba190c68-0a3d-4a87-9359-e3ae761327f4 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:33.330646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba190c68-0a3d-4a87-9359-e3ae761327f4 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:33.331995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 696/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:33.336905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bfa60-8fc1-424c-bae5-81cf6846b798 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:33.337314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bfa60-8fc1-424c-bae5-81cf6846b798 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:33.337472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bfa60-8fc1-424c-bae5-81cf6846b798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:33.358263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:33.415428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-420bfa60-8fc1-424c-bae5-81cf6846b798 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:33.416779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-420bfa60-8fc1-424c-bae5-81cf6846b798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:33.516140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 728/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:43:33.668124 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba190c68-0a3d-4a87-9359-e3ae761327f4 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:33.669657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ba190c68-0a3d-4a87-9359-e3ae761327f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:33.801526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:33.839180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba190c68-0a3d-4a87-9359-e3ae761327f4 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:33.951736 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad6c1b-c798-421f-b6be-720ffe992f4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:33.952398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad6c1b-c798-421f-b6be-720ffe992f4e service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:33.952683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad6c1b-c798-421f-b6be-720ffe992f4e service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:33.952960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad6c1b-c798-421f-b6be-720ffe992f4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:33.954570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-79ad6c1b-c798-421f-b6be-720ffe992f4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:34.030600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:34.044056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad6c1b-c798-421f-b6be-720ffe992f4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:34.045684 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad6c1b-c798-421f-b6be-720ffe992f4e service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:34.064372 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 697/1425] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:34.943224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b587c26-e13d-4c61-bb9c-cb84ba52caa0 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:34.943785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b587c26-e13d-4c61-bb9c-cb84ba52caa0 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:34.943785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b587c26-e13d-4c61-bb9c-cb84ba52caa0 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:34.945138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 729/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:43:34.951279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3e1d4-f62e-4ee0-8e40-861ccc20336a None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:34.951279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3e1d4-f62e-4ee0-8e40-861ccc20336a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:34.951826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3e1d4-f62e-4ee0-8e40-861ccc20336a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:35.026683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-9dc3e1d4-f62e-4ee0-8e40-861ccc20336a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:35.027959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9dc3e1d4-f62e-4ee0-8e40-861ccc20336a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:35.124520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 698/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:35.274013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b587c26-e13d-4c61-bb9c-cb84ba52caa0 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:35.275463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9b587c26-e13d-4c61-bb9c-cb84ba52caa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:35.359469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:35.420457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b587c26-e13d-4c61-bb9c-cb84ba52caa0 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:35.531361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f7777-74ce-408b-912b-4b4712d88cb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:35.531972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f7777-74ce-408b-912b-4b4712d88cb8 service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:35.532263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f7777-74ce-408b-912b-4b4712d88cb8 service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:35.532538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f7777-74ce-408b-912b-4b4712d88cb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:35.534140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-b72f7777-74ce-408b-912b-4b4712d88cb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:35.583502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:35.624718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f7777-74ce-408b-912b-4b4712d88cb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:35.626325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f7777-74ce-408b-912b-4b4712d88cb8 service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:35.644487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 730/1428] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:36.527531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703427ec-b93d-4c2e-b1e1-ff29cae618b2 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:36.528123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703427ec-b93d-4c2e-b1e1-ff29cae618b2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:36.528123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703427ec-b93d-4c2e-b1e1-ff29cae618b2 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:36.529464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 699/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:36.534637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57eb6b-66e4-4165-99d4-468fcb2517a4 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:36.535052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57eb6b-66e4-4165-99d4-468fcb2517a4 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:36.535249 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57eb6b-66e4-4165-99d4-468fcb2517a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:36.614776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9a57eb6b-66e4-4165-99d4-468fcb2517a4 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:36.616027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9a57eb6b-66e4-4165-99d4-468fcb2517a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:36.712366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 731/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:36.888749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703427ec-b93d-4c2e-b1e1-ff29cae618b2 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:36.890283 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-703427ec-b93d-4c2e-b1e1-ff29cae618b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:36.946244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:37.053575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703427ec-b93d-4c2e-b1e1-ff29cae618b2 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:37.184768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d49cb-cf96-4911-b980-dced44ac2d5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:37.185376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d49cb-cf96-4911-b980-dced44ac2d5c service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:37.186181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d49cb-cf96-4911-b980-dced44ac2d5c service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:37.186181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d49cb-cf96-4911-b980-dced44ac2d5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:37.187543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b45d49cb-cf96-4911-b980-dced44ac2d5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:37.194668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:37.278187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d49cb-cf96-4911-b980-dced44ac2d5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:37.279795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d49cb-cf96-4911-b980-dced44ac2d5c service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:37.298529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 700/1431] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:38.215704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbf2cb-da70-411b-9e6f-6c72683b20f4 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:38.216279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbf2cb-da70-411b-9e6f-6c72683b20f4 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:38.216392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbf2cb-da70-411b-9e6f-6c72683b20f4 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:38.217664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 732/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:38.222726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45c650-cc27-417f-9497-8737bb79c0f4 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:38.223185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45c650-cc27-417f-9497-8737bb79c0f4 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:38.223296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45c650-cc27-417f-9497-8737bb79c0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:38.297456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-3e45c650-cc27-417f-9497-8737bb79c0f4 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:38.298899 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3e45c650-cc27-417f-9497-8737bb79c0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:38.365301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:38.394082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 701/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:38.548573 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbf2cb-da70-411b-9e6f-6c72683b20f4 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:38.550095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a7cbf2cb-da70-411b-9e6f-6c72683b20f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:38.567566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:38.696541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbf2cb-da70-411b-9e6f-6c72683b20f4 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:38.784707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:38.813629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e3450-b726-401f-b0b0-1ad1811d652c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:38.814304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e3450-b726-401f-b0b0-1ad1811d652c service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:38.814595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e3450-b726-401f-b0b0-1ad1811d652c service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:38.814868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e3450-b726-401f-b0b0-1ad1811d652c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:38.816492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-dc1e3450-b726-401f-b0b0-1ad1811d652c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:38.906173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e3450-b726-401f-b0b0-1ad1811d652c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:38.907787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e3450-b726-401f-b0b0-1ad1811d652c service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:38.927013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 733/1434] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:39.836464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3685806-5519-419d-bc97-f3ad83f47397 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:39.837012 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3685806-5519-419d-bc97-f3ad83f47397 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:39.837012 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3685806-5519-419d-bc97-f3ad83f47397 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:39.838376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 702/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:39.843776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769932f3-45bd-45db-9b75-dd4a225e2d5b None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:39.844234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769932f3-45bd-45db-9b75-dd4a225e2d5b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:39.844370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769932f3-45bd-45db-9b75-dd4a225e2d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:39.927601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-769932f3-45bd-45db-9b75-dd4a225e2d5b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:39.928821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-769932f3-45bd-45db-9b75-dd4a225e2d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:39.969956 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:40.023483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 734/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:43:40.176108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3685806-5519-419d-bc97-f3ad83f47397 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:40.178896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c3685806-5519-419d-bc97-f3ad83f47397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:40.326938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3685806-5519-419d-bc97-f3ad83f47397 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:40.381487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:40.442268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2b04b-d787-4bea-bdb8-2a88ce4fb4b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:40.442854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2b04b-d787-4bea-bdb8-2a88ce4fb4b0 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:40.443184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2b04b-d787-4bea-bdb8-2a88ce4fb4b0 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:40.443467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2b04b-d787-4bea-bdb8-2a88ce4fb4b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:40.445126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-94c2b04b-d787-4bea-bdb8-2a88ce4fb4b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:40.535488 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2b04b-d787-4bea-bdb8-2a88ce4fb4b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:40.537115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2b04b-d787-4bea-bdb8-2a88ce4fb4b0 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:40.555227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 703/1437] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:41.420222 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd204d6b-74ef-4bf9-9a82-6723cc9ab74d None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:41.420820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd204d6b-74ef-4bf9-9a82-6723cc9ab74d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:41.420820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd204d6b-74ef-4bf9-9a82-6723cc9ab74d None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:41.422294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 735/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:41.427588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa30db9-fc37-4703-833a-e5103d44e43e None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:41.428033 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa30db9-fc37-4703-833a-e5103d44e43e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:41.428173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa30db9-fc37-4703-833a-e5103d44e43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:41.503955 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7fa30db9-fc37-4703-833a-e5103d44e43e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:41.505186 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7fa30db9-fc37-4703-833a-e5103d44e43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:41.507222 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:41.603413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 704/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:41.753611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd204d6b-74ef-4bf9-9a82-6723cc9ab74d None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:41.755118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-dd204d6b-74ef-4bf9-9a82-6723cc9ab74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:41.904666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd204d6b-74ef-4bf9-9a82-6723cc9ab74d None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:41.931070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:42.016521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b65aba-aeb1-48ab-bea8-c476a763aa17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:42.017034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b65aba-aeb1-48ab-bea8-c476a763aa17 service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:42.017307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b65aba-aeb1-48ab-bea8-c476a763aa17 service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:42.017613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b65aba-aeb1-48ab-bea8-c476a763aa17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:42.019233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a4b65aba-aeb1-48ab-bea8-c476a763aa17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:42.108535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b65aba-aeb1-48ab-bea8-c476a763aa17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:42.110115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b65aba-aeb1-48ab-bea8-c476a763aa17 service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:42.128224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 736/1440] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:43.955644 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1052ed0-8601-4bca-9335-ffca90a0bef6 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:43.956251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1052ed0-8601-4bca-9335-ffca90a0bef6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:43.956251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1052ed0-8601-4bca-9335-ffca90a0bef6 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:43.957642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 705/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:43.963492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da659bee-4bce-4f25-9d77-e32dae4a046a None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:43.963950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da659bee-4bce-4f25-9d77-e32dae4a046a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:43.964128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da659bee-4bce-4f25-9d77-e32dae4a046a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:44.005054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:44.029991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-da659bee-4bce-4f25-9d77-e32dae4a046a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:44.031207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-da659bee-4bce-4f25-9d77-e32dae4a046a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:44.127211 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 737/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:44.409646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1052ed0-8601-4bca-9335-ffca90a0bef6 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:44.411293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a1052ed0-8601-4bca-9335-ffca90a0bef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:44.518031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1052ed0-8601-4bca-9335-ffca90a0bef6 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:44.550327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:44.593501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b62ed-45e3-46ac-ab8c-67cf7a1c19f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:44.594116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b62ed-45e3-46ac-ab8c-67cf7a1c19f2 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:44.594396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b62ed-45e3-46ac-ab8c-67cf7a1c19f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:44.594734 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b62ed-45e3-46ac-ab8c-67cf7a1c19f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:44.596377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e74b62ed-45e3-46ac-ab8c-67cf7a1c19f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:44.705968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b62ed-45e3-46ac-ab8c-67cf7a1c19f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:44.707423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b62ed-45e3-46ac-ab8c-67cf7a1c19f2 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:44.726333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 706/1443] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:46.803448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024cd80-c3dc-4fdd-bd21-585d4f1f2bea None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:46.804006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024cd80-c3dc-4fdd-bd21-585d4f1f2bea None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:46.804006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024cd80-c3dc-4fdd-bd21-585d4f1f2bea None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:46.805380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 738/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:46.810718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6f8d0-199e-461a-b4a7-c714ba4345df None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:46.811158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6f8d0-199e-461a-b4a7-c714ba4345df None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:46.811262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6f8d0-199e-461a-b4a7-c714ba4345df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:46.876185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e7b6f8d0-199e-461a-b4a7-c714ba4345df None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:46.877330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e7b6f8d0-199e-461a-b4a7-c714ba4345df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:46.889971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:46.974759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 707/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:47.258992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024cd80-c3dc-4fdd-bd21-585d4f1f2bea None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:47.261120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9024cd80-c3dc-4fdd-bd21-585d4f1f2bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:47.365891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024cd80-c3dc-4fdd-bd21-585d4f1f2bea None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:47.380591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:47.442192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028d0c4-7d8b-4c18-8a83-c9efbcdcb895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:47.442802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028d0c4-7d8b-4c18-8a83-c9efbcdcb895 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:47.443089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028d0c4-7d8b-4c18-8a83-c9efbcdcb895 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:47.443369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028d0c4-7d8b-4c18-8a83-c9efbcdcb895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:47.445814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0028d0c4-7d8b-4c18-8a83-c9efbcdcb895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:47.554821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028d0c4-7d8b-4c18-8a83-c9efbcdcb895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:47.557332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028d0c4-7d8b-4c18-8a83-c9efbcdcb895 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:47.576840 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 739/1446] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:49.884079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51130f-cba7-4e5b-b4be-8248beeb830a None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:49.885144 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51130f-cba7-4e5b-b4be-8248beeb830a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:49.885238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51130f-cba7-4e5b-b4be-8248beeb830a None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:49.886584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 708/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:49.891943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7df09-b82c-4878-a17b-a84e9c868aaa None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:49.892364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7df09-b82c-4878-a17b-a84e9c868aaa None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:49.892504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7df09-b82c-4878-a17b-a84e9c868aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:49.941986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:49.960172 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-74b7df09-b82c-4878-a17b-a84e9c868aaa None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:49.961394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-74b7df09-b82c-4878-a17b-a84e9c868aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:50.075558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 740/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:50.362725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51130f-cba7-4e5b-b4be-8248beeb830a None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:50.364309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4d51130f-cba7-4e5b-b4be-8248beeb830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:50.469624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51130f-cba7-4e5b-b4be-8248beeb830a None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:50.511025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:50.545675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5c8fc-b362-4bf8-9dd8-99edec8573e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:50.546275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5c8fc-b362-4bf8-9dd8-99edec8573e0 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:50.546554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5c8fc-b362-4bf8-9dd8-99edec8573e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:50.546823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5c8fc-b362-4bf8-9dd8-99edec8573e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:50.548679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d4b5c8fc-b362-4bf8-9dd8-99edec8573e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:50.658438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5c8fc-b362-4bf8-9dd8-99edec8573e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:50.659971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5c8fc-b362-4bf8-9dd8-99edec8573e0 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:50.678905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 709/1449] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:52.691792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76f970-c2d8-40ad-829a-07f95badd466 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:52.692373 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76f970-c2d8-40ad-829a-07f95badd466 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:52.692373 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76f970-c2d8-40ad-829a-07f95badd466 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:52.693722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 741/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:52.699116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b754d1a-36da-4e8d-b5e8-884af7c172fa None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:52.699566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b754d1a-36da-4e8d-b5e8-884af7c172fa None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:52.699672 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b754d1a-36da-4e8d-b5e8-884af7c172fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:52.765337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-3b754d1a-36da-4e8d-b5e8-884af7c172fa None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:52.766546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3b754d1a-36da-4e8d-b5e8-884af7c172fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:52.787197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:52.866189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 710/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:53.150948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76f970-c2d8-40ad-829a-07f95badd466 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:53.152543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2a76f970-c2d8-40ad-829a-07f95badd466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:53.256395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76f970-c2d8-40ad-829a-07f95badd466 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:53.277727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:53.332319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a9fd7-1690-475c-aa57-715f73d83119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:53.332924 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a9fd7-1690-475c-aa57-715f73d83119 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:53.333225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a9fd7-1690-475c-aa57-715f73d83119 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:53.333477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a9fd7-1690-475c-aa57-715f73d83119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:53.335062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ac7a9fd7-1690-475c-aa57-715f73d83119 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:53.455115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a9fd7-1690-475c-aa57-715f73d83119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:53.456602 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a9fd7-1690-475c-aa57-715f73d83119 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:53.474903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 742/1452] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:55.438511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410c2d0-8ff3-4acc-8359-0e1e8aecd766 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:55.439095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410c2d0-8ff3-4acc-8359-0e1e8aecd766 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:55.439095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410c2d0-8ff3-4acc-8359-0e1e8aecd766 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:55.440482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 711/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:55.445904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d6545-cbc6-4773-81af-6f33923c93a6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:55.446356 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d6545-cbc6-4773-81af-6f33923c93a6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:55.446462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d6545-cbc6-4773-81af-6f33923c93a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:55.513628 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:55.516476 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3e4d6545-cbc6-4773-81af-6f33923c93a6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:55.517682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3e4d6545-cbc6-4773-81af-6f33923c93a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:55.614594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 743/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:55.900700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410c2d0-8ff3-4acc-8359-0e1e8aecd766 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:55.902437 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7410c2d0-8ff3-4acc-8359-0e1e8aecd766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:56.010065 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410c2d0-8ff3-4acc-8359-0e1e8aecd766 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:56.061206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:56.084931 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ddd33-0557-4809-ba71-8672a82cc7da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:56.085539 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ddd33-0557-4809-ba71-8672a82cc7da service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:56.085828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ddd33-0557-4809-ba71-8672a82cc7da service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:56.086093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ddd33-0557-4809-ba71-8672a82cc7da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:56.087914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9f4ddd33-0557-4809-ba71-8672a82cc7da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:56.196910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ddd33-0557-4809-ba71-8672a82cc7da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:56.198348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ddd33-0557-4809-ba71-8672a82cc7da service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:56.216993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 712/1455] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:43:58.176157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d5142-fa9a-43a0-a7a8-c0ff9ae20a0e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:58.176692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d5142-fa9a-43a0-a7a8-c0ff9ae20a0e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:58.176805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d5142-fa9a-43a0-a7a8-c0ff9ae20a0e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:58.178067 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 744/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:43:58.183801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a2b53-39b9-4f4c-a346-103a900b11ec None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:58.184326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a2b53-39b9-4f4c-a346-103a900b11ec None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:58.184443 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a2b53-39b9-4f4c-a346-103a900b11ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:58.252553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-347a2b53-39b9-4f4c-a346-103a900b11ec None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:43:58.253779 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-347a2b53-39b9-4f4c-a346-103a900b11ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:58.283183 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:58.351915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 713/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:43:58.637730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d5142-fa9a-43a0-a7a8-c0ff9ae20a0e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:43:58.639287 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-094d5142-fa9a-43a0-a7a8-c0ff9ae20a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:58.746093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d5142-fa9a-43a0-a7a8-c0ff9ae20a0e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:43:58.775151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:43:58.820546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ea383-355c-42b4-8564-f6a9bc805d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:43:58.821155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ea383-355c-42b4-8564-f6a9bc805d9d service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:43:58.821442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ea383-355c-42b4-8564-f6a9bc805d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:43:58.821720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ea383-355c-42b4-8564-f6a9bc805d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:43:58.823326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-b43ea383-355c-42b4-8564-f6a9bc805d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:43:58.934200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ea383-355c-42b4-8564-f6a9bc805d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:43:58.935690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ea383-355c-42b4-8564-f6a9bc805d9d service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:43:58.954149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 745/1458] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:43:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:44:00.925255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d36842-280e-4394-ab5b-38d28526b323 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:00.925785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d36842-280e-4394-ab5b-38d28526b323 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:00.925785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d36842-280e-4394-ab5b-38d28526b323 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:00.927071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 714/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:00.932899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba813d91-8db9-4e37-b830-429cbe42564e None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:00.933334 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba813d91-8db9-4e37-b830-429cbe42564e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:00.933560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba813d91-8db9-4e37-b830-429cbe42564e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:00.999551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ba813d91-8db9-4e37-b830-429cbe42564e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:01.000785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ba813d91-8db9-4e37-b830-429cbe42564e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:01.009724 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:01.100120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 746/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:44:01.384476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d36842-280e-4394-ab5b-38d28526b323 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:44:01.386026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-23d36842-280e-4394-ab5b-38d28526b323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:01.490240 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d36842-280e-4394-ab5b-38d28526b323 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:44:01.546282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:01.583917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d5bc9-8e01-49d1-b8ae-cf9e6d98e5fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:44:01.584530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d5bc9-8e01-49d1-b8ae-cf9e6d98e5fd service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:01.584810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d5bc9-8e01-49d1-b8ae-cf9e6d98e5fd service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:01.585080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d5bc9-8e01-49d1-b8ae-cf9e6d98e5fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:01.587605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cc1d5bc9-8e01-49d1-b8ae-cf9e6d98e5fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:01.699344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d5bc9-8e01-49d1-b8ae-cf9e6d98e5fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:44:01.700820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d5bc9-8e01-49d1-b8ae-cf9e6d98e5fd service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:44:01.718443 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 715/1461] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:44:03.688383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b799ccf-b8c4-484d-a04c-d1e9ab956121 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:03.688909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b799ccf-b8c4-484d-a04c-d1e9ab956121 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:03.689025 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b799ccf-b8c4-484d-a04c-d1e9ab956121 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:03.690318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 747/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:03.695697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad22bc-1392-482c-ada3-e07fcf121803 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:03.696116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad22bc-1392-482c-ada3-e07fcf121803 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:03.696862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad22bc-1392-482c-ada3-e07fcf121803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:03.762363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b7ad22bc-1392-482c-ada3-e07fcf121803 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:03.763514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b7ad22bc-1392-482c-ada3-e07fcf121803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:03.801815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:03.860552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 716/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:44:04.145188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b799ccf-b8c4-484d-a04c-d1e9ab956121 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:44:04.146831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0b799ccf-b8c4-484d-a04c-d1e9ab956121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:04.250278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b799ccf-b8c4-484d-a04c-d1e9ab956121 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:44:04.286756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:04.322117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845df386-244c-4074-b5d9-13b326ce66a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:44:04.322716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845df386-244c-4074-b5d9-13b326ce66a8 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:04.322996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845df386-244c-4074-b5d9-13b326ce66a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:04.323271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845df386-244c-4074-b5d9-13b326ce66a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:04.325531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-845df386-244c-4074-b5d9-13b326ce66a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:04.433247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845df386-244c-4074-b5d9-13b326ce66a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:44:04.434708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845df386-244c-4074-b5d9-13b326ce66a8 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:44:04.452791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 748/1464] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:44:06.391404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44e842-bd85-4934-ac86-a60358bc51d1 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:06.391988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44e842-bd85-4934-ac86-a60358bc51d1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:06.391988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44e842-bd85-4934-ac86-a60358bc51d1 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:06.393393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 717/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:06.398935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfb4ed-f3a8-4ffe-8894-e5f15198eb85 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:06.399394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfb4ed-f3a8-4ffe-8894-e5f15198eb85 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:06.399490 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfb4ed-f3a8-4ffe-8894-e5f15198eb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:06.467020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-d0bfb4ed-f3a8-4ffe-8894-e5f15198eb85 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:06.468265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-d0bfb4ed-f3a8-4ffe-8894-e5f15198eb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:06.485967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:06.563637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 749/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:44:06.847493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44e842-bd85-4934-ac86-a60358bc51d1 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:44:06.849167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5a44e842-bd85-4934-ac86-a60358bc51d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:06.952732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44e842-bd85-4934-ac86-a60358bc51d1 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:44:07.018833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:07.026206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277f0cb-7ca9-4fba-9d37-781c0f2e5bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:44:07.031029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277f0cb-7ca9-4fba-9d37-781c0f2e5bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:07.031029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277f0cb-7ca9-4fba-9d37-781c0f2e5bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:07.031029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277f0cb-7ca9-4fba-9d37-781c0f2e5bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:07.031029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f277f0cb-7ca9-4fba-9d37-781c0f2e5bf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:07.138397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277f0cb-7ca9-4fba-9d37-781c0f2e5bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:44:07.139844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277f0cb-7ca9-4fba-9d37-781c0f2e5bf2 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:44:07.158423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 718/1467] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:44:09.099170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18929559-344e-4da9-a95f-57c54a3c74fd None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:09.099712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18929559-344e-4da9-a95f-57c54a3c74fd None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:09.099835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18929559-344e-4da9-a95f-57c54a3c74fd None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:09.101164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 750/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:09.106452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620c6f6-0980-4bf7-b7c1-5c4c0138dca3 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:09.106884 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620c6f6-0980-4bf7-b7c1-5c4c0138dca3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:09.107019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620c6f6-0980-4bf7-b7c1-5c4c0138dca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:09.174224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7620c6f6-0980-4bf7-b7c1-5c4c0138dca3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:09.175507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7620c6f6-0980-4bf7-b7c1-5c4c0138dca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:09.224037 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:09.274282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 719/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:44:09.559235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18929559-344e-4da9-a95f-57c54a3c74fd None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:44:09.560816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-18929559-344e-4da9-a95f-57c54a3c74fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:09.664202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18929559-344e-4da9-a95f-57c54a3c74fd None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:44:09.712449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:09.738658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b759ae-4ddc-4a75-ba9d-a823e2268070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:44:09.739266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b759ae-4ddc-4a75-ba9d-a823e2268070 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:09.739549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b759ae-4ddc-4a75-ba9d-a823e2268070 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:09.739826 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b759ae-4ddc-4a75-ba9d-a823e2268070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:09.741462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-05b759ae-4ddc-4a75-ba9d-a823e2268070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:09.849348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b759ae-4ddc-4a75-ba9d-a823e2268070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:44:09.850791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b759ae-4ddc-4a75-ba9d-a823e2268070 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:44:09.869778 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 751/1470] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:44:19.270913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=22 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:19.867330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=66 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:29.587526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a4f25-6e0e-48cc-a4f0-5da82076085c None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:29.588107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a4f25-6e0e-48cc-a4f0-5da82076085c None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:29.588107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a4f25-6e0e-48cc-a4f0-5da82076085c None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:29.589450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 720/1471] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:44:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:29.594841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5ee25-d24b-4ea6-a9aa-bd62733097c1 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:29.595281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5ee25-d24b-4ea6-a9aa-bd62733097c1 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:29.595418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5ee25-d24b-4ea6-a9aa-bd62733097c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:29.661627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0fb5ee25-d24b-4ea6-a9aa-bd62733097c1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:29.662853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0fb5ee25-d24b-4ea6-a9aa-bd62733097c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:29.760506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 752/1472] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:44:30.057397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7e555-4dd7-45f5-a025-1339a444bbb3 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:30.057694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7e555-4dd7-45f5-a025-1339a444bbb3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:30.057979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7e555-4dd7-45f5-a025-1339a444bbb3 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:30.059309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 721/1473] 10.222.0.98 () {54 vars in 967 bytes} [Fri Aug 26 15:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:30.064108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1077d-0f0b-4346-89f9-9e0a27182f53 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:30.064402 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1077d-0f0b-4346-89f9-9e0a27182f53 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:30.064682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1077d-0f0b-4346-89f9-9e0a27182f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:30.143884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-36f1077d-0f0b-4346-89f9-9e0a27182f53 None None] MFA Rules not processed for user `cef193a54eff44f2b53716e3ec1a29cf`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:30.145130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-36f1077d-0f0b-4346-89f9-9e0a27182f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:30.175447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:30.234187 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 753/1474] 10.222.0.98 () {58 vars in 1064 bytes} [Fri Aug 26 15:44:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:44:30.242909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7e555-4dd7-45f5-a025-1339a444bbb3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:44:30.244512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-95d7e555-4dd7-45f5-a025-1339a444bbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:30.374553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7e555-4dd7-45f5-a025-1339a444bbb3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:44:30.454979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd99cd-2eda-4ca1-94f6-988d95e5cf6f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cef193a54eff44f2b53716e3ec1a29cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:44:30.455993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd99cd-2eda-4ca1-94f6-988d95e5cf6f service glance] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:30.456303 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd99cd-2eda-4ca1-94f6-988d95e5cf6f service glance] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:30.456580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd99cd-2eda-4ca1-94f6-988d95e5cf6f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:30.458177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c8bd99cd-2eda-4ca1-94f6-988d95e5cf6f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:30.529238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:30.587180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd99cd-2eda-4ca1-94f6-988d95e5cf6f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:44:30.588650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd99cd-2eda-4ca1-94f6-988d95e5cf6f service glance] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:44:30.607855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 722/1475] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:44:32.643627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ab271-3fcc-4698-8933-73327b767fb2 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:32.644176 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ab271-3fcc-4698-8933-73327b767fb2 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:32.644303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ab271-3fcc-4698-8933-73327b767fb2 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:32.645541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 754/1476] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:32.651611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e56e3-6522-4d8d-ad15-83bd46a1f699 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:32.652033 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e56e3-6522-4d8d-ad15-83bd46a1f699 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:32.652288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e56e3-6522-4d8d-ad15-83bd46a1f699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:32.719192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-887e56e3-6522-4d8d-ad15-83bd46a1f699 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:32.720452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-887e56e3-6522-4d8d-ad15-83bd46a1f699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:32.817196 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 723/1477] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:44:33.096432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ab271-3fcc-4698-8933-73327b767fb2 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:44:33.097999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-af6ab271-3fcc-4698-8933-73327b767fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:33.222957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ab271-3fcc-4698-8933-73327b767fb2 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:44:33.271578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:33.316876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8c373-e1e2-4cfd-b8d6-6a6276d42282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:44:33.317482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8c373-e1e2-4cfd-b8d6-6a6276d42282 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:33.317767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8c373-e1e2-4cfd-b8d6-6a6276d42282 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:33.318042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8c373-e1e2-4cfd-b8d6-6a6276d42282 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:33.318678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8c373-e1e2-4cfd-b8d6-6a6276d42282 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:44:33.320525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: WARNING py.warnings [None req-4bf8c373-e1e2-4cfd-b8d6-6a6276d42282 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:44:33.320525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: warnings.warn(msg) Aug 26 15:44:33.320525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: Aug 26 15:44:33.321698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8c373-e1e2-4cfd-b8d6-6a6276d42282 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:44:33.343257 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 755/1478] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Aug 26 15:44:33 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:44:35.286417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3886b-b7e9-4226-ab9d-f46618017612 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:35.286955 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3886b-b7e9-4226-ab9d-f46618017612 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:35.286955 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3886b-b7e9-4226-ab9d-f46618017612 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:35.288351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 724/1479] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:35.293748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470c272-a90b-4c91-83a8-31fb4aff1413 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:35.294176 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470c272-a90b-4c91-83a8-31fb4aff1413 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:35.294310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470c272-a90b-4c91-83a8-31fb4aff1413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:35.359663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-4470c272-a90b-4c91-83a8-31fb4aff1413 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:35.360909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4470c272-a90b-4c91-83a8-31fb4aff1413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:35.458668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 756/1480] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:44:35.759236 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3886b-b7e9-4226-ab9d-f46618017612 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:44:35.760787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c6b3886b-b7e9-4226-ab9d-f46618017612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:35.770560 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:35.912705 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3886b-b7e9-4226-ab9d-f46618017612 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:44:35.993920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:36.026018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd4529-fecc-44c2-851a-f9b021a1f21f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:44:36.026628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4529-fecc-44c2-851a-f9b021a1f21f service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:36.026906 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4529-fecc-44c2-851a-f9b021a1f21f service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:36.027176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4529-fecc-44c2-851a-f9b021a1f21f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:36.028793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-afcd4529-fecc-44c2-851a-f9b021a1f21f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:36.120331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd4529-fecc-44c2-851a-f9b021a1f21f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:44:36.121915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd4529-fecc-44c2-851a-f9b021a1f21f service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:44:36.140215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 725/1481] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:44:38.258088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb9c7-17de-4620-9a67-1b4f5dae329b None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:38.258682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb9c7-17de-4620-9a67-1b4f5dae329b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:38.258682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb9c7-17de-4620-9a67-1b4f5dae329b None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:38.260045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 757/1482] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:38.265548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200434-0767-4d55-bc19-0704b98c91e5 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:38.265987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200434-0767-4d55-bc19-0704b98c91e5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:38.278431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200434-0767-4d55-bc19-0704b98c91e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:38.333590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-8b200434-0767-4d55-bc19-0704b98c91e5 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:38.334849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8b200434-0767-4d55-bc19-0704b98c91e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:38.384876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:38.447938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 726/1483] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:44:38.753669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeb9c7-17de-4620-9a67-1b4f5dae329b None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:44:38.755250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1ebeb9c7-17de-4620-9a67-1b4f5dae329b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:38.780069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:38.860865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeb9c7-17de-4620-9a67-1b4f5dae329b None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:44:38.932901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180850c-f924-4900-ae4c-6a1d93038a06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:44:38.933501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180850c-f924-4900-ae4c-6a1d93038a06 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:38.933783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180850c-f924-4900-ae4c-6a1d93038a06 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:38.934056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180850c-f924-4900-ae4c-6a1d93038a06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:38.935630 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3180850c-f924-4900-ae4c-6a1d93038a06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:39.009022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:39.045457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180850c-f924-4900-ae4c-6a1d93038a06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:44:39.046890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180850c-f924-4900-ae4c-6a1d93038a06 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:44:39.065054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 758/1484] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:44:41.075452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9881b1-7993-439b-9e7f-28d34e14719c None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:41.076146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9881b1-7993-439b-9e7f-28d34e14719c None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:41.076312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9881b1-7993-439b-9e7f-28d34e14719c None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:41.077907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 727/1485] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:44:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:41.085181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ebad7-f3ca-4fb2-99a5-7633d38151f1 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:41.085736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ebad7-f3ca-4fb2-99a5-7633d38151f1 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:41.085834 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ebad7-f3ca-4fb2-99a5-7633d38151f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:41.159498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-879ebad7-f3ca-4fb2-99a5-7633d38151f1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:41.160841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-879ebad7-f3ca-4fb2-99a5-7633d38151f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:41.259480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 759/1486] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:44:41.557947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9881b1-7993-439b-9e7f-28d34e14719c None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:44:41.559490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cd9881b1-7993-439b-9e7f-28d34e14719c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:41.689214 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9881b1-7993-439b-9e7f-28d34e14719c None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:44:41.774255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:41.776284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0e0f4-8c8e-4ba0-892d-e95766387823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:44:41.776892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0e0f4-8c8e-4ba0-892d-e95766387823 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:41.777176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0e0f4-8c8e-4ba0-892d-e95766387823 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:41.777452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0e0f4-8c8e-4ba0-892d-e95766387823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:41.779050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cfb0e0f4-8c8e-4ba0-892d-e95766387823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:41.892515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0e0f4-8c8e-4ba0-892d-e95766387823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:44:41.893971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0e0f4-8c8e-4ba0-892d-e95766387823 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:44:41.914653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 728/1487] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:44:44.025219 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a11652-2d5d-4ea3-83f8-43cffc72f31a None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:44.025797 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a11652-2d5d-4ea3-83f8-43cffc72f31a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:44.025907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a11652-2d5d-4ea3-83f8-43cffc72f31a None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:44.027177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 760/1488] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:44.032577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decf36d-d696-4ea3-a179-389082cb3789 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:44.032982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decf36d-d696-4ea3-a179-389082cb3789 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:44.033143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decf36d-d696-4ea3-a179-389082cb3789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:44.098107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-6decf36d-d696-4ea3-a179-389082cb3789 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:44.099317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6decf36d-d696-4ea3-a179-389082cb3789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:44.151775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:44.194725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 729/1489] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:44:44.490481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a11652-2d5d-4ea3-83f8-43cffc72f31a None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:44:44.492549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-83a11652-2d5d-4ea3-83f8-43cffc72f31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:44.540210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:44.610063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a11652-2d5d-4ea3-83f8-43cffc72f31a None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:44:44.684510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065e6f2-ffa9-4145-a85a-551be9708983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:44:44.685116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065e6f2-ffa9-4145-a85a-551be9708983 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:44.685403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065e6f2-ffa9-4145-a85a-551be9708983 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:44.685679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065e6f2-ffa9-4145-a85a-551be9708983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:44.687283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1065e6f2-ffa9-4145-a85a-551be9708983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:44.767428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:44.796404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065e6f2-ffa9-4145-a85a-551be9708983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:44:44.798225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065e6f2-ffa9-4145-a85a-551be9708983 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:44:44.817524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 761/1490] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:44:47.094246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5496ac-75bb-4023-94db-3e012349d134 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:47.094817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5496ac-75bb-4023-94db-3e012349d134 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:47.094817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5496ac-75bb-4023-94db-3e012349d134 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:47.096257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 730/1491] 10.222.0.98 () {54 vars in 927 bytes} [Fri Aug 26 15:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:44:47.102120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1acd7f-8a0a-4df2-bd6b-6171f1feb8ae None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:47.102553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1acd7f-8a0a-4df2-bd6b-6171f1feb8ae None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:47.102662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1acd7f-8a0a-4df2-bd6b-6171f1feb8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:47.169145 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-4b1acd7f-8a0a-4df2-bd6b-6171f1feb8ae None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:44:47.170361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4b1acd7f-8a0a-4df2-bd6b-6171f1feb8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:47.266453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 762/1492] 10.222.0.98 () {58 vars in 1024 bytes} [Fri Aug 26 15:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:44:47.562484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5496ac-75bb-4023-94db-3e012349d134 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:44:47.564090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3e5496ac-75bb-4023-94db-3e012349d134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:47.669316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5496ac-75bb-4023-94db-3e012349d134 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:44:47.742444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb067f-c0fa-47f6-a9b7-80e10ca6ab20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:44:47.743049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb067f-c0fa-47f6-a9b7-80e10ca6ab20 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:44:47.743334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb067f-c0fa-47f6-a9b7-80e10ca6ab20 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:44:47.743613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb067f-c0fa-47f6-a9b7-80e10ca6ab20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:44:47.745260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-85eb067f-c0fa-47f6-a9b7-80e10ca6ab20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:44:47.754307 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:47.857264 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb067f-c0fa-47f6-a9b7-80e10ca6ab20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:44:47.858613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb067f-c0fa-47f6-a9b7-80e10ca6ab20 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:44:47.876829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 731/1493] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:44:57.264567 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=82 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:44:57.874646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=50 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:23.852409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb440102-7a7a-49ef-8a9f-64a59200f0c1 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:23.862856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb440102-7a7a-49ef-8a9f-64a59200f0c1 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:23.862856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb440102-7a7a-49ef-8a9f-64a59200f0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:23.933131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-fb440102-7a7a-49ef-8a9f-64a59200f0c1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:46:23.934348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-fb440102-7a7a-49ef-8a9f-64a59200f0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:24.032725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 763/1494] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:46:24.041638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c90e17-cb6a-473c-bcf2-a2cb582f1bf1 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:24.042181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c90e17-cb6a-473c-bcf2-a2cb582f1bf1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:24.042181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c90e17-cb6a-473c-bcf2-a2cb582f1bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:24.122643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-00c90e17-cb6a-473c-bcf2-a2cb582f1bf1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:46:24.124526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-00c90e17-cb6a-473c-bcf2-a2cb582f1bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:24.223607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 732/1495] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:46:24.229142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb440102-7a7a-49ef-8a9f-64a59200f0c1 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:46:24.230380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-fb440102-7a7a-49ef-8a9f-64a59200f0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:24.243069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb440102-7a7a-49ef-8a9f-64a59200f0c1 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:46:24.305868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:24.340455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4cfeb-a6b9-4eb1-9dee-219a535c0308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:46:24.341066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4cfeb-a6b9-4eb1-9dee-219a535c0308 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:24.341346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4cfeb-a6b9-4eb1-9dee-219a535c0308 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:24.341624 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4cfeb-a6b9-4eb1-9dee-219a535c0308 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:24.342279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4cfeb-a6b9-4eb1-9dee-219a535c0308 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:46:24.342583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4cfeb-a6b9-4eb1-9dee-219a535c0308 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:46:24.343633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: WARNING py.warnings [None req-e9c4cfeb-a6b9-4eb1-9dee-219a535c0308 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:46:24.343633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: warnings.warn(msg) Aug 26 15:46:24.343633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: Aug 26 15:46:24.344559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4cfeb-a6b9-4eb1-9dee-219a535c0308 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:46:24.357061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 764/1496] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:46:24.365960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c90e17-cb6a-473c-bcf2-a2cb582f1bf1 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:46:24.375508 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c90e17-cb6a-473c-bcf2-a2cb582f1bf1 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:46:24.438500 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:24.476711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448f81-9ea3-4fe9-83e6-2f3982428386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:46:24.477102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448f81-9ea3-4fe9-83e6-2f3982428386 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:24.477388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448f81-9ea3-4fe9-83e6-2f3982428386 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:24.477660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448f81-9ea3-4fe9-83e6-2f3982428386 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:24.478294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08448f81-9ea3-4fe9-83e6-2f3982428386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:46:24.480081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: WARNING py.warnings [None req-08448f81-9ea3-4fe9-83e6-2f3982428386 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:46:24.480081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: warnings.warn(msg) Aug 26 15:46:24.480081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: Aug 26 15:46:24.481279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08448f81-9ea3-4fe9-83e6-2f3982428386 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:46:24.685036 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 733/1497] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:46:24 2022] POST /identity/v3/projects => generated 379 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:46:24.689874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4cfeb-a6b9-4eb1-9dee-219a535c0308 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:46:24.691440 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e9c4cfeb-a6b9-4eb1-9dee-219a535c0308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:24.726582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4cfeb-a6b9-4eb1-9dee-219a535c0308 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:46:24.823700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a84c18-ba63-4ce0-8874-2912c4d5b284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:46:24.824355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a84c18-ba63-4ce0-8874-2912c4d5b284 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:24.824635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a84c18-ba63-4ce0-8874-2912c4d5b284 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:24.824908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a84c18-ba63-4ce0-8874-2912c4d5b284 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:24.825538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a84c18-ba63-4ce0-8874-2912c4d5b284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:46:24.826572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: WARNING py.warnings [None req-a3a84c18-ba63-4ce0-8874-2912c4d5b284 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:46:24.826572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: warnings.warn(msg) Aug 26 15:46:24.826572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: Aug 26 15:46:24.827757 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a84c18-ba63-4ce0-8874-2912c4d5b284 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:46:24.981138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 765/1498] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:46:24 2022] POST /identity/v3/users => generated 290 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:46:24.985242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448f81-9ea3-4fe9-83e6-2f3982428386 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:46:24.986771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-08448f81-9ea3-4fe9-83e6-2f3982428386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:25.054176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448f81-9ea3-4fe9-83e6-2f3982428386 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:46:25.141251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:25.152563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c01d2-edc4-41c3-bdac-d7c9000d4279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:46:25.153174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c01d2-edc4-41c3-bdac-d7c9000d4279 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:25.153458 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c01d2-edc4-41c3-bdac-d7c9000d4279 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:25.153732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c01d2-edc4-41c3-bdac-d7c9000d4279 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:25.154375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c01d2-edc4-41c3-bdac-d7c9000d4279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:46:25.155389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: WARNING py.warnings [None req-0e8c01d2-edc4-41c3-bdac-d7c9000d4279 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:46:25.155389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: warnings.warn(msg) Aug 26 15:46:25.155389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: Aug 26 15:46:25.156304 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c01d2-edc4-41c3-bdac-d7c9000d4279 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:46:25.168927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 734/1499] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:46:25.173167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a84c18-ba63-4ce0-8874-2912c4d5b284 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:46:25.174975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a3a84c18-ba63-4ce0-8874-2912c4d5b284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:25.186772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a84c18-ba63-4ce0-8874-2912c4d5b284 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:46:25.214438 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:25.305208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265017e5-3fa1-4fda-afab-9b3867dcb1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:46:25.305741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265017e5-3fa1-4fda-afab-9b3867dcb1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:25.306023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265017e5-3fa1-4fda-afab-9b3867dcb1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:25.306296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265017e5-3fa1-4fda-afab-9b3867dcb1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:25.306942 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265017e5-3fa1-4fda-afab-9b3867dcb1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:46:25.307985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: WARNING py.warnings [None req-265017e5-3fa1-4fda-afab-9b3867dcb1c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:46:25.307985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: warnings.warn(msg) Aug 26 15:46:25.307985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: Aug 26 15:46:25.308868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265017e5-3fa1-4fda-afab-9b3867dcb1c0 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:46:25.322799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 766/1500] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:46:25.327125 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c01d2-edc4-41c3-bdac-d7c9000d4279 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:46:25.328792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0e8c01d2-edc4-41c3-bdac-d7c9000d4279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:25.363323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c01d2-edc4-41c3-bdac-d7c9000d4279 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:46:25.460348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39269b9-0bed-4b4b-ad24-1c8ed0e4985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:46:25.461011 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39269b9-0bed-4b4b-ad24-1c8ed0e4985a admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:25.461292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39269b9-0bed-4b4b-ad24-1c8ed0e4985a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:25.461566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39269b9-0bed-4b4b-ad24-1c8ed0e4985a admin admin] PATH_INFO: `/v3/projects/7559eccd562c4b628e9055ebaef27dc5/users/eadc372b01d9465ebab134c14ae04a1c/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:25.484446 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39269b9-0bed-4b4b-ad24-1c8ed0e4985a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7559eccd562c4b628e9055ebaef27dc5, user_id=eadc372b01d9465ebab134c14ae04a1c, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:46:25.485484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: WARNING py.warnings [None req-c39269b9-0bed-4b4b-ad24-1c8ed0e4985a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:46:25.485484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: warnings.warn(msg) Aug 26 15:46:25.485484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: Aug 26 15:46:25.487433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39269b9-0bed-4b4b-ad24-1c8ed0e4985a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:46:25.584147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 735/1501] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:46:25 2022] PUT /identity/v3/projects/7559eccd562c4b628e9055ebaef27dc5/users/eadc372b01d9465ebab134c14ae04a1c/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:46:25.596504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4cb8e-2e9e-465b-8e14-50ad844517de None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:25.596857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4cb8e-2e9e-465b-8e14-50ad844517de None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:25.597129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4cb8e-2e9e-465b-8e14-50ad844517de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:25.656415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-bad4cb8e-2e9e-465b-8e14-50ad844517de None None] MFA Rules not processed for user `eadc372b01d9465ebab134c14ae04a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:46:25.657652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bad4cb8e-2e9e-465b-8e14-50ad844517de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:25.739549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 767/1502] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:46:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:46:25.752704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0327b-d002-49de-a3f2-02bc999e92fb None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:25.752991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0327b-d002-49de-a3f2-02bc999e92fb None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:25.753265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0327b-d002-49de-a3f2-02bc999e92fb None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:25.754598 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 736/1503] 10.222.0.98 () {54 vars in 964 bytes} [Fri Aug 26 15:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:46:25.759850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45198b1a-27a7-4d08-af4c-85a4d6916e48 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:25.760172 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45198b1a-27a7-4d08-af4c-85a4d6916e48 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:25.760452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45198b1a-27a7-4d08-af4c-85a4d6916e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:25.793170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:25.842605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-45198b1a-27a7-4d08-af4c-85a4d6916e48 None None] MFA Rules not processed for user `3a1ff8aff9e048c6972843e0b50c4d45`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:46:25.843750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-45198b1a-27a7-4d08-af4c-85a4d6916e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:25.921949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 768/1504] 10.222.0.98 () {58 vars in 1061 bytes} [Fri Aug 26 15:46:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:46:25.930693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0327b-d002-49de-a3f2-02bc999e92fb None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:46:25.932478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ded0327b-d002-49de-a3f2-02bc999e92fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:25.984447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:26.041186 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0327b-d002-49de-a3f2-02bc999e92fb None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:46:26.142123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e21f81-68f4-449f-a971-bb2ae2adf260 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:46:26.142813 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e21f81-68f4-449f-a971-bb2ae2adf260 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:26.142919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e21f81-68f4-449f-a971-bb2ae2adf260 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:26.143192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e21f81-68f4-449f-a971-bb2ae2adf260 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:26.144881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f1e21f81-68f4-449f-a971-bb2ae2adf260 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:26.250071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e21f81-68f4-449f-a971-bb2ae2adf260 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:46:26.251515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e21f81-68f4-449f-a971-bb2ae2adf260 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:46:26.255005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:26.269234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 737/1505] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:46:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:46:26.316794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654b3f-a693-4200-abd5-3e0396b34bb9 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:26.317228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654b3f-a693-4200-abd5-3e0396b34bb9 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:26.317357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654b3f-a693-4200-abd5-3e0396b34bb9 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:26.318640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 769/1506] 10.222.0.98 () {52 vars in 830 bytes} [Fri Aug 26 15:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:46:26.323412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e21f81-68f4-449f-a971-bb2ae2adf260 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:46:26.324608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f1e21f81-68f4-449f-a971-bb2ae2adf260 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:26.424191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e21f81-68f4-449f-a971-bb2ae2adf260 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:46:26.518079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:46:26.518707 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:26.518988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:26.519260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] PATH_INFO: `/v3/users/eadc372b01d9465ebab134c14ae04a1c` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:26.532139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eadc372b01d9465ebab134c14ae04a1c)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:46:26.533165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: WARNING py.warnings [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:46:26.533165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: warnings.warn(msg) Aug 26 15:46:26.533165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: Aug 26 15:46:26.534371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:46:26.544015 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:26.779490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eadc372b01d9465ebab134c14ae04a1c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:46:26.780066 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eadc372b01d9465ebab134c14ae04a1c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:46:26.806227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eadc372b01d9465ebab134c14ae04a1c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:46:26.905183 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eadc372b01d9465ebab134c14ae04a1c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:46:26.921921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eadc372b01d9465ebab134c14ae04a1c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:46:26.928292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 738/1507] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:46:26 2022] DELETE /identity/v3/users/eadc372b01d9465ebab134c14ae04a1c => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:46:26.932660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df378e3e-9755-47c6-a6b0-9c0a847aad8c None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:26.932946 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df378e3e-9755-47c6-a6b0-9c0a847aad8c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:26.933226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df378e3e-9755-47c6-a6b0-9c0a847aad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:26.991648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-df378e3e-9755-47c6-a6b0-9c0a847aad8c None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:46:26.995396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-df378e3e-9755-47c6-a6b0-9c0a847aad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:27.066908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:27.080010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 770/1508] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:46:27.090831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:46:27.092134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:27.194414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0774723-2f5e-47a7-b72e-3d6862695cb6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:46:27.248589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:27.267589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2bfa25-d73c-4f48-979e-fa95c39c8afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:46:27.268239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bfa25-d73c-4f48-979e-fa95c39c8afc service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:27.268526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bfa25-d73c-4f48-979e-fa95c39c8afc service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:27.268806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bfa25-d73c-4f48-979e-fa95c39c8afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:27.270409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5d2bfa25-d73c-4f48-979e-fa95c39c8afc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:27.379621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2bfa25-d73c-4f48-979e-fa95c39c8afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:46:27.381530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2bfa25-d73c-4f48-979e-fa95c39c8afc service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:46:27.400421 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 739/1509] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:46:27.964619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df378e3e-9755-47c6-a6b0-9c0a847aad8c None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:46:27.965866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-df378e3e-9755-47c6-a6b0-9c0a847aad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:46:27.977602 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df378e3e-9755-47c6-a6b0-9c0a847aad8c None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:46:28.076695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:46:28.077321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:46:28.077600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:46:28.077890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] PATH_INFO: `/v3/projects/7559eccd562c4b628e9055ebaef27dc5` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:46:28.088275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7559eccd562c4b628e9055ebaef27dc5)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:46:28.089303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: WARNING py.warnings [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:46:28.089303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: warnings.warn(msg) Aug 26 15:46:28.089303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: Aug 26 15:46:28.090484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:46:28.214944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7559eccd562c4b628e9055ebaef27dc5', 'request_id': 'req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:46:28.318275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7559eccd562c4b628e9055ebaef27dc5', 'request_id': 'req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:46:28.332711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7559eccd562c4b628e9055ebaef27dc5', 'request_id': 'req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:46:28.354884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] The token cache is being invalidate because project 7559eccd562c4b628e9055ebaef27dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:46:28.363197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:46:28.363197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-7a14b4f3-b8b7-46cb-9874-b307f32e04c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:46:28.363197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 771/1510] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:46:27 2022] DELETE /identity/v3/projects/7559eccd562c4b628e9055ebaef27dc5 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:46:37.454497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=62 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:46:38.352750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:05.096710 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1de2c-d7fd-4690-a430-6a5fd39671b4 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:05.097320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1de2c-d7fd-4690-a430-6a5fd39671b4 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:05.097320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1de2c-d7fd-4690-a430-6a5fd39671b4 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:05.098600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 740/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:05.103728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c02a2-2582-4475-9928-7920ae3f77ca None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:05.104288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c02a2-2582-4475-9928-7920ae3f77ca None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:05.104392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c02a2-2582-4475-9928-7920ae3f77ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:05.187960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0e6c02a2-2582-4475-9928-7920ae3f77ca None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:05.189202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0e6c02a2-2582-4475-9928-7920ae3f77ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:05.295712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 772/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:05.453881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1de2c-d7fd-4690-a430-6a5fd39671b4 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:05.456071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-bae1de2c-d7fd-4690-a430-6a5fd39671b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:05.577099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1de2c-d7fd-4690-a430-6a5fd39671b4 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:05.582659 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbceb1-d11d-4377-9528-ba07369e58ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:05.583271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbceb1-d11d-4377-9528-ba07369e58ff service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:05.583549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbceb1-d11d-4377-9528-ba07369e58ff service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:05.583822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbceb1-d11d-4377-9528-ba07369e58ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:05.592856 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbceb1-d11d-4377-9528-ba07369e58ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:05.594440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbceb1-d11d-4377-9528-ba07369e58ff service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:05.603975 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 741/1513] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:06.457085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3b3d1-1255-4e70-971d-9a5e7da12c26 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:06.457648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3b3d1-1255-4e70-971d-9a5e7da12c26 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:06.457753 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3b3d1-1255-4e70-971d-9a5e7da12c26 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:06.459162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 773/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:06.464113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d967f-9886-4d27-acf3-110a413e639d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:06.464526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d967f-9886-4d27-acf3-110a413e639d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:06.464680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d967f-9886-4d27-acf3-110a413e639d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:06.484825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ad1d967f-9886-4d27-acf3-110a413e639d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:06.486048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ad1d967f-9886-4d27-acf3-110a413e639d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:06.494089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 742/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:06.647692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3b3d1-1255-4e70-971d-9a5e7da12c26 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:06.662510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3b3d1-1255-4e70-971d-9a5e7da12c26 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:06.668102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71560ac-9467-4d14-a65b-b7a4ad9fd641 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:06.668727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71560ac-9467-4d14-a65b-b7a4ad9fd641 service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:06.669010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71560ac-9467-4d14-a65b-b7a4ad9fd641 service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:06.669283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71560ac-9467-4d14-a65b-b7a4ad9fd641 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:06.677776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71560ac-9467-4d14-a65b-b7a4ad9fd641 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:06.679411 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71560ac-9467-4d14-a65b-b7a4ad9fd641 service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:06.688967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 774/1516] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:07.305206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa4dba-3382-4a0d-852b-a08041994704 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:07.305493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa4dba-3382-4a0d-852b-a08041994704 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:07.305773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa4dba-3382-4a0d-852b-a08041994704 None None] PATH_INFO: `/v3` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:07.307061 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 743/1517] 10.222.0.67 () {54 vars in 925 bytes} [Fri Aug 26 15:48:07 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:48:07.340600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f12706-9e32-4e18-b1d2-f6f227e677ee None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:07.340893 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f12706-9e32-4e18-b1d2-f6f227e677ee None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:07.341178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f12706-9e32-4e18-b1d2-f6f227e677ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:07.399387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-25f12706-9e32-4e18-b1d2-f6f227e677ee None None] MFA Rules not processed for user `b28f987c90df4c19a12bb1819dc65cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:07.400646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-25f12706-9e32-4e18-b1d2-f6f227e677ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:07.417667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:07.471584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 775/1518] 10.222.0.67 () {58 vars in 1015 bytes} [Fri Aug 26 15:48:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:16.479389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=56 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:16.873867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ca986-9c46-4d5e-bf86-01f2e119fc05 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:16.874070 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ca986-9c46-4d5e-bf86-01f2e119fc05 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:16.874334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ca986-9c46-4d5e-bf86-01f2e119fc05 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:16.876338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 744/1519] 10.222.0.98 () {54 vars in 961 bytes} [Fri Aug 26 15:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:16.881278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1ae16-0e3b-4ad2-a5c1-69146abe36be None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:16.881766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1ae16-0e3b-4ad2-a5c1-69146abe36be None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:16.881869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1ae16-0e3b-4ad2-a5c1-69146abe36be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:16.901924 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-c9c1ae16-0e3b-4ad2-a5c1-69146abe36be None None] MFA Rules not processed for user `b28f987c90df4c19a12bb1819dc65cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:16.903147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c9c1ae16-0e3b-4ad2-a5c1-69146abe36be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:16.911814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 776/1520] 10.222.0.98 () {58 vars in 1058 bytes} [Fri Aug 26 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:16.920461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ca986-9c46-4d5e-bf86-01f2e119fc05 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:16.935429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ca986-9c46-4d5e-bf86-01f2e119fc05 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:16.940924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac9a34-1e8f-493f-afd1-f07010bc90c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:16.941533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac9a34-1e8f-493f-afd1-f07010bc90c0 service placement] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:16.941813 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac9a34-1e8f-493f-afd1-f07010bc90c0 service placement] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:16.942086 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac9a34-1e8f-493f-afd1-f07010bc90c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:16.950937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac9a34-1e8f-493f-afd1-f07010bc90c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:16.952574 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac9a34-1e8f-493f-afd1-f07010bc90c0 service placement] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:16.962439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 745/1521] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:18.003661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0b3c4-092a-49b3-a85d-4475b38631ba None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:18.004299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0b3c4-092a-49b3-a85d-4475b38631ba None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:18.004299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0b3c4-092a-49b3-a85d-4475b38631ba None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:18.005596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 777/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:18.010709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eefbf-c84a-46aa-9a2a-f147ced99e31 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:18.013178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eefbf-c84a-46aa-9a2a-f147ced99e31 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:18.013178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eefbf-c84a-46aa-9a2a-f147ced99e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:18.031389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-fc3eefbf-c84a-46aa-9a2a-f147ced99e31 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:18.032638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-fc3eefbf-c84a-46aa-9a2a-f147ced99e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:18.040484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 746/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:18.197802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0b3c4-092a-49b3-a85d-4475b38631ba None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:18.213921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0b3c4-092a-49b3-a85d-4475b38631ba None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:18.219681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a48b3-3af6-4c6b-ad51-d0b30736e73d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:18.220362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a48b3-3af6-4c6b-ad51-d0b30736e73d service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:18.220681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a48b3-3af6-4c6b-ad51-d0b30736e73d service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:18.221860 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a48b3-3af6-4c6b-ad51-d0b30736e73d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:18.230551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a48b3-3af6-4c6b-ad51-d0b30736e73d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:18.232377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a48b3-3af6-4c6b-ad51-d0b30736e73d service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:18.246179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 778/1524] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:19.077023 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f971b7-c2ce-4383-98e2-079a787b101c None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:19.077589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f971b7-c2ce-4383-98e2-079a787b101c None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:19.077589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f971b7-c2ce-4383-98e2-079a787b101c None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:19.078927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 747/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:19.083589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed2bf3-1c23-444f-b35a-516a957c90e6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:19.084088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed2bf3-1c23-444f-b35a-516a957c90e6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:19.084183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed2bf3-1c23-444f-b35a-516a957c90e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:19.103241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-71ed2bf3-1c23-444f-b35a-516a957c90e6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:19.104550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-71ed2bf3-1c23-444f-b35a-516a957c90e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:19.112630 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 779/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:48:19.271330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f971b7-c2ce-4383-98e2-079a787b101c None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:19.285912 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f971b7-c2ce-4383-98e2-079a787b101c None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:19.291312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a64ec-a5a3-4fd1-bde1-d9efb33e24a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:19.291940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a64ec-a5a3-4fd1-bde1-d9efb33e24a9 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:19.292755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a64ec-a5a3-4fd1-bde1-d9efb33e24a9 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:19.293034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a64ec-a5a3-4fd1-bde1-d9efb33e24a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:19.301030 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a64ec-a5a3-4fd1-bde1-d9efb33e24a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:19.302604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a64ec-a5a3-4fd1-bde1-d9efb33e24a9 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:19.312925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 748/1527] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:23.221063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3042e-2f8a-455f-92a6-5b90ef3187fc None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:23.221714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3042e-2f8a-455f-92a6-5b90ef3187fc None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:23.221714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3042e-2f8a-455f-92a6-5b90ef3187fc None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:23.222984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 780/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:23.228281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f72b29-fd87-4cef-a152-35d703de9f6d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:23.232598 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f72b29-fd87-4cef-a152-35d703de9f6d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:23.232598 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f72b29-fd87-4cef-a152-35d703de9f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:23.249579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b7f72b29-fd87-4cef-a152-35d703de9f6d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:23.250841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b7f72b29-fd87-4cef-a152-35d703de9f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:23.259099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 749/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:23.542734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3042e-2f8a-455f-92a6-5b90ef3187fc None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:23.544476 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4ac3042e-2f8a-455f-92a6-5b90ef3187fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:23.625714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3042e-2f8a-455f-92a6-5b90ef3187fc None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:23.629350 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f7296-7dad-4729-852e-bd68b281ae5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:23.629960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f7296-7dad-4729-852e-bd68b281ae5c service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:23.630243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f7296-7dad-4729-852e-bd68b281ae5c service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:23.630526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f7296-7dad-4729-852e-bd68b281ae5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:23.640609 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f7296-7dad-4729-852e-bd68b281ae5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:23.642041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f7296-7dad-4729-852e-bd68b281ae5c service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:23.651399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 781/1530] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:25.555506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9950a8-852f-454e-a352-5dfdccbf1bf2 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:25.556095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9950a8-852f-454e-a352-5dfdccbf1bf2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:25.556095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9950a8-852f-454e-a352-5dfdccbf1bf2 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:25.557447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 750/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:25.562713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf7006-3acb-4c67-aaa2-d6c2a46a2d3e None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:25.563143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf7006-3acb-4c67-aaa2-d6c2a46a2d3e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:25.563274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf7006-3acb-4c67-aaa2-d6c2a46a2d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:25.582611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-2ebf7006-3acb-4c67-aaa2-d6c2a46a2d3e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:25.583821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2ebf7006-3acb-4c67-aaa2-d6c2a46a2d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:25.592282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 782/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:25.875542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9950a8-852f-454e-a352-5dfdccbf1bf2 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:25.887023 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9950a8-852f-454e-a352-5dfdccbf1bf2 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:25.890550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2cd0a-6e81-427b-847d-536e6ec6b907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:25.891144 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2cd0a-6e81-427b-847d-536e6ec6b907 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:25.891424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2cd0a-6e81-427b-847d-536e6ec6b907 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:25.891696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2cd0a-6e81-427b-847d-536e6ec6b907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:25.900371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2cd0a-6e81-427b-847d-536e6ec6b907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:25.902479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2cd0a-6e81-427b-847d-536e6ec6b907 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:25.913563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 751/1533] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:29.355955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eda742-7c3d-4855-9cf6-e765b732b5ec None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:29.356529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eda742-7c3d-4855-9cf6-e765b732b5ec None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:29.356641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eda742-7c3d-4855-9cf6-e765b732b5ec None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:29.357901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 783/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:29.363097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d7211-1424-4396-82e5-d07060263985 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:29.363558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d7211-1424-4396-82e5-d07060263985 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:29.363664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d7211-1424-4396-82e5-d07060263985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:29.383268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ef7d7211-1424-4396-82e5-d07060263985 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:29.384514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ef7d7211-1424-4396-82e5-d07060263985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:29.393573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 752/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:29.950381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eda742-7c3d-4855-9cf6-e765b732b5ec None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:29.952095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f9eda742-7c3d-4855-9cf6-e765b732b5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:30.018090 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:30.031572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eda742-7c3d-4855-9cf6-e765b732b5ec None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:30.035359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31649e7-b5df-42d2-ad42-40bab91103c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:30.036013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31649e7-b5df-42d2-ad42-40bab91103c0 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:30.036299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31649e7-b5df-42d2-ad42-40bab91103c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:30.036708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31649e7-b5df-42d2-ad42-40bab91103c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:30.046736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31649e7-b5df-42d2-ad42-40bab91103c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:30.048228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31649e7-b5df-42d2-ad42-40bab91103c0 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:30.057742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 784/1536] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:31.994775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d1ea2-8bab-40b6-ab92-21678017d7aa None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:31.995371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d1ea2-8bab-40b6-ab92-21678017d7aa None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:31.995371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d1ea2-8bab-40b6-ab92-21678017d7aa None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:31.996950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 753/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:32.002081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791b28f-7d98-4673-a167-20b7c70a25b7 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:32.002490 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791b28f-7d98-4673-a167-20b7c70a25b7 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:32.002632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791b28f-7d98-4673-a167-20b7c70a25b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:32.022346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0791b28f-7d98-4673-a167-20b7c70a25b7 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:32.023530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0791b28f-7d98-4673-a167-20b7c70a25b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:32.032234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 785/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:32.583185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d1ea2-8bab-40b6-ab92-21678017d7aa None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:32.584925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-410d1ea2-8bab-40b6-ab92-21678017d7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:32.598600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d1ea2-8bab-40b6-ab92-21678017d7aa None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:32.602865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec07e05-431e-40f8-ba20-c20156dd228e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:32.603579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec07e05-431e-40f8-ba20-c20156dd228e service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:32.603762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec07e05-431e-40f8-ba20-c20156dd228e service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:32.604088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec07e05-431e-40f8-ba20-c20156dd228e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:32.615308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec07e05-431e-40f8-ba20-c20156dd228e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:32.616800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec07e05-431e-40f8-ba20-c20156dd228e service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:32.627323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 754/1539] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:42.017195 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=11 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:42.622797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=33 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:58.689095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba3371-a102-48a4-9aaf-7477baa33912 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:58.689737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba3371-a102-48a4-9aaf-7477baa33912 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:58.689737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba3371-a102-48a4-9aaf-7477baa33912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:58.717044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0aba3371-a102-48a4-9aaf-7477baa33912 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:58.718292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0aba3371-a102-48a4-9aaf-7477baa33912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:58.729783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 786/1540] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:58.739342 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a18dd3-db86-4070-b6ff-afdc01319c5f None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:58.739993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a18dd3-db86-4070-b6ff-afdc01319c5f None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:58.739993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a18dd3-db86-4070-b6ff-afdc01319c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:58.769544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-49a18dd3-db86-4070-b6ff-afdc01319c5f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:58.770805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-49a18dd3-db86-4070-b6ff-afdc01319c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:58.788320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 755/1541] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:58.793431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba3371-a102-48a4-9aaf-7477baa33912 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:58.811381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba3371-a102-48a4-9aaf-7477baa33912 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:58.822741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208f4fd-288b-47c7-b911-688b8b8812af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:58.823647 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208f4fd-288b-47c7-b911-688b8b8812af admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:58.823954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208f4fd-288b-47c7-b911-688b8b8812af admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:58.824233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208f4fd-288b-47c7-b911-688b8b8812af admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:58.824894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208f4fd-288b-47c7-b911-688b8b8812af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:49:58.825198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208f4fd-288b-47c7-b911-688b8b8812af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:58.827455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208f4fd-288b-47c7-b911-688b8b8812af admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:58.847083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 787/1542] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:49:58.855476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a18dd3-db86-4070-b6ff-afdc01319c5f None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:58.866405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a18dd3-db86-4070-b6ff-afdc01319c5f None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:58.876948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc49288-14d6-49c9-917f-757bb5750523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:58.877673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc49288-14d6-49c9-917f-757bb5750523 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:58.877967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc49288-14d6-49c9-917f-757bb5750523 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:58.878250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc49288-14d6-49c9-917f-757bb5750523 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:58.878910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc49288-14d6-49c9-917f-757bb5750523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:58.880917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc49288-14d6-49c9-917f-757bb5750523 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:59.003310 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 756/1543] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:49:58 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:49:59.008095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208f4fd-288b-47c7-b911-688b8b8812af admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:59.022307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208f4fd-288b-47c7-b911-688b8b8812af admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:59.115676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea29df-7c7b-47ea-9e66-50c45a963792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:59.116716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea29df-7c7b-47ea-9e66-50c45a963792 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.117230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea29df-7c7b-47ea-9e66-50c45a963792 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.118028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea29df-7c7b-47ea-9e66-50c45a963792 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.118895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea29df-7c7b-47ea-9e66-50c45a963792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:59.121048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4552f9f-e725-4a0a-b4b5-3a5fb94b8348 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.121342 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4552f9f-e725-4a0a-b4b5-3a5fb94b8348 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.121628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4552f9f-e725-4a0a-b4b5-3a5fb94b8348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.121877 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea29df-7c7b-47ea-9e66-50c45a963792 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:59.158128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e4552f9f-e725-4a0a-b4b5-3a5fb94b8348 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.159366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e4552f9f-e725-4a0a-b4b5-3a5fb94b8348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.170665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 757/1544] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.172022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce5a62-6d3b-487d-ab3d-3e62b71fb980 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.172313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce5a62-6d3b-487d-ab3d-3e62b71fb980 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.172595 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce5a62-6d3b-487d-ab3d-3e62b71fb980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.205747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-5bce5a62-6d3b-487d-ab3d-3e62b71fb980 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.207009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5bce5a62-6d3b-487d-ab3d-3e62b71fb980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.217594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 758/1545] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.218849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc6489-1912-4241-b676-84fe6af192b9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.219144 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc6489-1912-4241-b676-84fe6af192b9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.219425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc6489-1912-4241-b676-84fe6af192b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.225128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 788/1546] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:49:59.229743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af6856-86b3-465d-8919-c47c7150ad88 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.229743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af6856-86b3-465d-8919-c47c7150ad88 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.229743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af6856-86b3-465d-8919-c47c7150ad88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.258304 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-65fc6489-1912-4241-b676-84fe6af192b9 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.259543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-65fc6489-1912-4241-b676-84fe6af192b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.270749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-a8af6856-86b3-465d-8919-c47c7150ad88 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.273317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a8af6856-86b3-465d-8919-c47c7150ad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.276764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 759/1547] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.277431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc6489-1912-4241-b676-84fe6af192b9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:59.291169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 789/1548] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.292480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565d0a3-de53-47ed-a07e-fd5e6801cf83 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.292768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565d0a3-de53-47ed-a07e-fd5e6801cf83 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.293117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565d0a3-de53-47ed-a07e-fd5e6801cf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.299474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc6489-1912-4241-b676-84fe6af192b9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:59.308383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470c0e0-9033-4026-8b75-5dbd827cea8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:59.309019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470c0e0-9033-4026-8b75-5dbd827cea8d admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.309306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470c0e0-9033-4026-8b75-5dbd827cea8d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.309587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470c0e0-9033-4026-8b75-5dbd827cea8d admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.313113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4470c0e0-9033-4026-8b75-5dbd827cea8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:59.315156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4470c0e0-9033-4026-8b75-5dbd827cea8d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:59.331915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 760/1549] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:49:59.331915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59b73-e66c-4c76-b7e9-4ad0ec82c141 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.331915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59b73-e66c-4c76-b7e9-4ad0ec82c141 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.331915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59b73-e66c-4c76-b7e9-4ad0ec82c141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.332822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-6565d0a3-de53-47ed-a07e-fd5e6801cf83 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.334622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6565d0a3-de53-47ed-a07e-fd5e6801cf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.346228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 790/1550] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.347421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565d0a3-de53-47ed-a07e-fd5e6801cf83 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:59.349820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-6fa59b73-e66c-4c76-b7e9-4ad0ec82c141 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.351025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6fa59b73-e66c-4c76-b7e9-4ad0ec82c141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.357032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565d0a3-de53-47ed-a07e-fd5e6801cf83 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:59.359653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 761/1551] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.361017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190803f-a2a8-4ab2-8fe2-3c8363669782 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.361313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190803f-a2a8-4ab2-8fe2-3c8363669782 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.361592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190803f-a2a8-4ab2-8fe2-3c8363669782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.366163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd88bc-8bd6-45e7-b57c-56ed181ad0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:59.367107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd88bc-8bd6-45e7-b57c-56ed181ad0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.367401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd88bc-8bd6-45e7-b57c-56ed181ad0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.367717 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd88bc-8bd6-45e7-b57c-56ed181ad0c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.368414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd88bc-8bd6-45e7-b57c-56ed181ad0c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:49:59.368727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd88bc-8bd6-45e7-b57c-56ed181ad0c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:59.370409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd88bc-8bd6-45e7-b57c-56ed181ad0c8 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:59.381298 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c190803f-a2a8-4ab2-8fe2-3c8363669782 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.382532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c190803f-a2a8-4ab2-8fe2-3c8363669782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.390455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 791/1552] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:49:59.390455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d10a9-9ee7-4797-ace9-eed8c9771984 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.390455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d10a9-9ee7-4797-ace9-eed8c9771984 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.390455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d10a9-9ee7-4797-ace9-eed8c9771984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.394700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 762/1553] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.396045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9faf9-e2d2-46d3-868b-2b38d47a05f2 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.396333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9faf9-e2d2-46d3-868b-2b38d47a05f2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.396609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9faf9-e2d2-46d3-868b-2b38d47a05f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.404623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-115d10a9-9ee7-4797-ace9-eed8c9771984 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.405879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-115d10a9-9ee7-4797-ace9-eed8c9771984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.413844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 792/1554] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.414627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d10a9-9ee7-4797-ace9-eed8c9771984 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:59.416020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c4c9faf9-e2d2-46d3-868b-2b38d47a05f2 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.417218 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c4c9faf9-e2d2-46d3-868b-2b38d47a05f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.424341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d10a9-9ee7-4797-ace9-eed8c9771984 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:59.425208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 763/1555] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.426553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163a3ba-289e-4558-a70b-091b68654299 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.426846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163a3ba-289e-4558-a70b-091b68654299 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.427122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163a3ba-289e-4558-a70b-091b68654299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.429754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b97aad-fa1c-48b5-a4c6-28763a688538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:59.430377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b97aad-fa1c-48b5-a4c6-28763a688538 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.430661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b97aad-fa1c-48b5-a4c6-28763a688538 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.430940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b97aad-fa1c-48b5-a4c6-28763a688538 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.431853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b97aad-fa1c-48b5-a4c6-28763a688538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:59.433613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b97aad-fa1c-48b5-a4c6-28763a688538 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:59.444647 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 793/1556] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:49:59.445316 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b97aad-fa1c-48b5-a4c6-28763a688538 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:59.446981 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-9163a3ba-289e-4558-a70b-091b68654299 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.448312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9163a3ba-289e-4558-a70b-091b68654299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.456193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b97aad-fa1c-48b5-a4c6-28763a688538 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:59.456821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 764/1557] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.458029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852f8b1-404f-417b-916f-32b326412a3e None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.458321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852f8b1-404f-417b-916f-32b326412a3e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.458601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852f8b1-404f-417b-916f-32b326412a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.461235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18614b22-ef29-4580-b3a0-9b42526077a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:59.461840 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18614b22-ef29-4580-b3a0-9b42526077a4 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.462130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18614b22-ef29-4580-b3a0-9b42526077a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.462415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18614b22-ef29-4580-b3a0-9b42526077a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.463340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18614b22-ef29-4580-b3a0-9b42526077a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:49:59.463658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18614b22-ef29-4580-b3a0-9b42526077a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:59.465441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18614b22-ef29-4580-b3a0-9b42526077a4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:59.478492 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-0852f8b1-404f-417b-916f-32b326412a3e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.479181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 794/1558] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:49:59.479696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0852f8b1-404f-417b-916f-32b326412a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.480000 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18614b22-ef29-4580-b3a0-9b42526077a4 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:59.490285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 765/1559] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.491193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852f8b1-404f-417b-916f-32b326412a3e None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:59.496803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18614b22-ef29-4580-b3a0-9b42526077a4 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:59.502068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852f8b1-404f-417b-916f-32b326412a3e None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:59.502626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec4a36-c682-4f7d-b55f-005008e97c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:59.503371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec4a36-c682-4f7d-b55f-005008e97c04 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.503659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec4a36-c682-4f7d-b55f-005008e97c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.503970 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec4a36-c682-4f7d-b55f-005008e97c04 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.504597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec4a36-c682-4f7d-b55f-005008e97c04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:59.505695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: WARNING py.warnings [None req-b2ec4a36-c682-4f7d-b55f-005008e97c04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:49:59.505695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: warnings.warn(msg) Aug 26 15:49:59.505695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: Aug 26 15:49:59.506637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0a7e2-bfc8-4149-bbda-e5a31ecb1792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:59.506884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec4a36-c682-4f7d-b55f-005008e97c04 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:59.507329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0a7e2-bfc8-4149-bbda-e5a31ecb1792 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.507619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0a7e2-bfc8-4149-bbda-e5a31ecb1792 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.507918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0a7e2-bfc8-4149-bbda-e5a31ecb1792 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.508572 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0a7e2-bfc8-4149-bbda-e5a31ecb1792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:49:59.508878 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0a7e2-bfc8-4149-bbda-e5a31ecb1792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:59.509888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: WARNING py.warnings [None req-2dd0a7e2-bfc8-4149-bbda-e5a31ecb1792 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:49:59.509888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: warnings.warn(msg) Aug 26 15:49:59.509888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: Aug 26 15:49:59.510765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0a7e2-bfc8-4149-bbda-e5a31ecb1792 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:59.523915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 766/1560] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:49:59.524115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96353ff-938a-4d8b-954c-b4cad85ba161 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.524389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96353ff-938a-4d8b-954c-b4cad85ba161 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.524668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96353ff-938a-4d8b-954c-b4cad85ba161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.545222 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e96353ff-938a-4d8b-954c-b4cad85ba161 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.546405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e96353ff-938a-4d8b-954c-b4cad85ba161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.554070 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 767/1561] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.555284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2307b9-0fa1-4516-8117-8094e9d0f2eb None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.555561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2307b9-0fa1-4516-8117-8094e9d0f2eb None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.555833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2307b9-0fa1-4516-8117-8094e9d0f2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.574285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-8b2307b9-0fa1-4516-8117-8094e9d0f2eb None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.575459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8b2307b9-0fa1-4516-8117-8094e9d0f2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.584197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 768/1562] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.584991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2307b9-0fa1-4516-8117-8094e9d0f2eb None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:59.594982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2307b9-0fa1-4516-8117-8094e9d0f2eb None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:59.599550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce3709-6556-4236-835f-0274840c2dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:59.600281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3709-6556-4236-835f-0274840c2dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.600559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3709-6556-4236-835f-0274840c2dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.600830 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3709-6556-4236-835f-0274840c2dd6 admin admin] PATH_INFO: `/v3/projects/bc372e59524442debba38c57df458071/users/6861433b9f4441fe842b506def255c58/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.621306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce3709-6556-4236-835f-0274840c2dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc372e59524442debba38c57df458071, user_id=6861433b9f4441fe842b506def255c58, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:59.624003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce3709-6556-4236-835f-0274840c2dd6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:59.636221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 795/1563] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/projects => generated 365 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:49:59.636221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e284ccc-0889-46ff-a70d-0ae79df45106 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.636221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e284ccc-0889-46ff-a70d-0ae79df45106 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.636221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e284ccc-0889-46ff-a70d-0ae79df45106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.649844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-5e284ccc-0889-46ff-a70d-0ae79df45106 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:59.651035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5e284ccc-0889-46ff-a70d-0ae79df45106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:59.783277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 769/1564] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:49:59 2022] PUT /identity/v3/projects/bc372e59524442debba38c57df458071/users/6861433b9f4441fe842b506def255c58/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:49:59.784158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce3709-6556-4236-835f-0274840c2dd6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:59.844059 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:59.850054 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce3709-6556-4236-835f-0274840c2dd6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:59.850948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850ecfb-b7f6-4625-a100-9b234bb04261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:59.851560 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850ecfb-b7f6-4625-a100-9b234bb04261 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.851846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850ecfb-b7f6-4625-a100-9b234bb04261 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.852158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850ecfb-b7f6-4625-a100-9b234bb04261 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.852813 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850ecfb-b7f6-4625-a100-9b234bb04261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:59.854717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850ecfb-b7f6-4625-a100-9b234bb04261 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:59.859028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 796/1565] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:59.860080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e284ccc-0889-46ff-a70d-0ae79df45106 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:59.869820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e284ccc-0889-46ff-a70d-0ae79df45106 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:59.870311 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:59.875435 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5b8c2-fcfc-4d0c-b3fe-591ced6072e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:59.876019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5b8c2-fcfc-4d0c-b3fe-591ced6072e0 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.876304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5b8c2-fcfc-4d0c-b3fe-591ced6072e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.876581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5b8c2-fcfc-4d0c-b3fe-591ced6072e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.877229 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5b8c2-fcfc-4d0c-b3fe-591ced6072e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:49:59.877534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5b8c2-fcfc-4d0c-b3fe-591ced6072e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:59.879392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5b8c2-fcfc-4d0c-b3fe-591ced6072e0 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:59.891119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 797/1566] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:49:59.892091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5b8c2-fcfc-4d0c-b3fe-591ced6072e0 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:59.901896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5b8c2-fcfc-4d0c-b3fe-591ced6072e0 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:59.906710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21182c-d27b-4656-806e-3e1909994bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:59.907317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21182c-d27b-4656-806e-3e1909994bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:59.907604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21182c-d27b-4656-806e-3e1909994bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:59.907905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21182c-d27b-4656-806e-3e1909994bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:59.908525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21182c-d27b-4656-806e-3e1909994bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:59.910401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21182c-d27b-4656-806e-3e1909994bd6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.094916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 770/1567] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/projects => generated 379 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:00.095763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850ecfb-b7f6-4625-a100-9b234bb04261 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.155819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 798/1568] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:49:59 2022] POST /identity/v3/projects => generated 371 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:00.156825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21182c-d27b-4656-806e-3e1909994bd6 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.159314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850ecfb-b7f6-4625-a100-9b234bb04261 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.219152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d3b05-e576-4567-b20c-28d27429c8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.219757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d3b05-e576-4567-b20c-28d27429c8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.219927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21182c-d27b-4656-806e-3e1909994bd6 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.220051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d3b05-e576-4567-b20c-28d27429c8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.220327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d3b05-e576-4567-b20c-28d27429c8a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.221058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d3b05-e576-4567-b20c-28d27429c8a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:00.221363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d3b05-e576-4567-b20c-28d27429c8a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.223012 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d3b05-e576-4567-b20c-28d27429c8a0 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.224634 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ba566-b20a-4d83-884b-4082573daeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.225243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ba566-b20a-4d83-884b-4082573daeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.225529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ba566-b20a-4d83-884b-4082573daeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.225809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ba566-b20a-4d83-884b-4082573daeb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.226490 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ba566-b20a-4d83-884b-4082573daeb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:00.226798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ba566-b20a-4d83-884b-4082573daeb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.228432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ba566-b20a-4d83-884b-4082573daeb7 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.242217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 771/1569] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:00.242217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d3b05-e576-4567-b20c-28d27429c8a0 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.245919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d3b05-e576-4567-b20c-28d27429c8a0 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.249105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 799/1570] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:00.250922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbad63f-0a43-4f75-b653-d7ebb4678d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.251595 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbad63f-0a43-4f75-b653-d7ebb4678d60 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.251898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbad63f-0a43-4f75-b653-d7ebb4678d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.259339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbad63f-0a43-4f75-b653-d7ebb4678d60 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.259464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3003-7081-4c96-8936-2fd46430d357 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.259464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3003-7081-4c96-8936-2fd46430d357 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.259676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbad63f-0a43-4f75-b653-d7ebb4678d60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.259789 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3003-7081-4c96-8936-2fd46430d357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.259913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: WARNING py.warnings [None req-2cbad63f-0a43-4f75-b653-d7ebb4678d60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:50:00.259913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: warnings.warn(msg) Aug 26 15:50:00.259913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: Aug 26 15:50:00.259913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbad63f-0a43-4f75-b653-d7ebb4678d60 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.276742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-2fca3003-7081-4c96-8936-2fd46430d357 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:00.277970 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2fca3003-7081-4c96-8936-2fd46430d357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:00.288125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 800/1571] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:00.297026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca3003-7081-4c96-8936-2fd46430d357 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.307407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca3003-7081-4c96-8936-2fd46430d357 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.314057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4684cc6-6c8e-4e75-aeb9-570ac39fa44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.314753 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4684cc6-6c8e-4e75-aeb9-570ac39fa44a admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.315038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4684cc6-6c8e-4e75-aeb9-570ac39fa44a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.315318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4684cc6-6c8e-4e75-aeb9-570ac39fa44a admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.316011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4684cc6-6c8e-4e75-aeb9-570ac39fa44a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:00.316319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4684cc6-6c8e-4e75-aeb9-570ac39fa44a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.318000 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4684cc6-6c8e-4e75-aeb9-570ac39fa44a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.330534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 801/1572] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:00.331308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4684cc6-6c8e-4e75-aeb9-570ac39fa44a admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.342412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4684cc6-6c8e-4e75-aeb9-570ac39fa44a admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.347302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99eb2b-bae7-4668-b07c-48ae9458aa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.347922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99eb2b-bae7-4668-b07c-48ae9458aa3d admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.348346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99eb2b-bae7-4668-b07c-48ae9458aa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.348630 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99eb2b-bae7-4668-b07c-48ae9458aa3d admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.349255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99eb2b-bae7-4668-b07c-48ae9458aa3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.351157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99eb2b-bae7-4668-b07c-48ae9458aa3d admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.385331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 772/1573] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:00 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:00.386302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbad63f-0a43-4f75-b653-d7ebb4678d60 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.396448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbad63f-0a43-4f75-b653-d7ebb4678d60 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.401090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a6d53-421b-4abb-be6e-3c4cb7af149f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.401679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a6d53-421b-4abb-be6e-3c4cb7af149f admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.401966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a6d53-421b-4abb-be6e-3c4cb7af149f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.402247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a6d53-421b-4abb-be6e-3c4cb7af149f admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.402866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a6d53-421b-4abb-be6e-3c4cb7af149f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.405516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a6d53-421b-4abb-be6e-3c4cb7af149f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.453089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 802/1574] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:00 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:00.453901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99eb2b-bae7-4668-b07c-48ae9458aa3d admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.515248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99eb2b-bae7-4668-b07c-48ae9458aa3d admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.520271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f299d86-ed0b-4a0f-82fc-d711c3c79870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.520872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f299d86-ed0b-4a0f-82fc-d711c3c79870 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.521155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f299d86-ed0b-4a0f-82fc-d711c3c79870 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.521431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f299d86-ed0b-4a0f-82fc-d711c3c79870 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.522051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f299d86-ed0b-4a0f-82fc-d711c3c79870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.524043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f299d86-ed0b-4a0f-82fc-d711c3c79870 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.548032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 773/1575] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:00 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:00.548032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a6d53-421b-4abb-be6e-3c4cb7af149f admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.559398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a6d53-421b-4abb-be6e-3c4cb7af149f admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.564412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb150098-8572-46e8-9df5-e27eb13fe90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.565048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150098-8572-46e8-9df5-e27eb13fe90a admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.565362 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150098-8572-46e8-9df5-e27eb13fe90a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.565666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150098-8572-46e8-9df5-e27eb13fe90a admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.566322 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb150098-8572-46e8-9df5-e27eb13fe90a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.568270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb150098-8572-46e8-9df5-e27eb13fe90a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.637081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:00.643725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 803/1576] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:00 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:00.645039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f299d86-ed0b-4a0f-82fc-d711c3c79870 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.655382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f299d86-ed0b-4a0f-82fc-d711c3c79870 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.660571 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce478b3-03cf-471c-b3a6-788199b760bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.661499 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce478b3-03cf-471c-b3a6-788199b760bb admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.661794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce478b3-03cf-471c-b3a6-788199b760bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.662068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce478b3-03cf-471c-b3a6-788199b760bb admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.662692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce478b3-03cf-471c-b3a6-788199b760bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.664641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce478b3-03cf-471c-b3a6-788199b760bb admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.677966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 774/1577] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:00 2022] POST /identity/v3/projects => generated 393 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:00.678809 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb150098-8572-46e8-9df5-e27eb13fe90a admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.720787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:00.734238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb150098-8572-46e8-9df5-e27eb13fe90a admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.737970 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696dec3c-8ba4-482d-bdd2-ef5e5990f720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.738580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696dec3c-8ba4-482d-bdd2-ef5e5990f720 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.738859 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696dec3c-8ba4-482d-bdd2-ef5e5990f720 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.739161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696dec3c-8ba4-482d-bdd2-ef5e5990f720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.764110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 804/1578] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:00 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:00.764941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce478b3-03cf-471c-b3a6-788199b760bb admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.801973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696dec3c-8ba4-482d-bdd2-ef5e5990f720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.803416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696dec3c-8ba4-482d-bdd2-ef5e5990f720 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.813241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 775/1579] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:00.814116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696dec3c-8ba4-482d-bdd2-ef5e5990f720 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.823105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696dec3c-8ba4-482d-bdd2-ef5e5990f720 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.827558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce478b3-03cf-471c-b3a6-788199b760bb admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.827909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c942ec9-a38c-46d6-9390-b159b8ae08f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.828517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c942ec9-a38c-46d6-9390-b159b8ae08f4 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.828799 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c942ec9-a38c-46d6-9390-b159b8ae08f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.829075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c942ec9-a38c-46d6-9390-b159b8ae08f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.829719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c942ec9-a38c-46d6-9390-b159b8ae08f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.831292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c942ec9-a38c-46d6-9390-b159b8ae08f4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.832254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e456a-a879-401a-91f9-18018c6992a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.832869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e456a-a879-401a-91f9-18018c6992a8 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.833148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e456a-a879-401a-91f9-18018c6992a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.833433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e456a-a879-401a-91f9-18018c6992a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.834330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e456a-a879-401a-91f9-18018c6992a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.836029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e456a-a879-401a-91f9-18018c6992a8 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:00.845589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 776/1580] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:00.845589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c942ec9-a38c-46d6-9390-b159b8ae08f4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:00.853840 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c942ec9-a38c-46d6-9390-b159b8ae08f4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:00.859213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2fe1c-32d1-4dcf-847e-b9f538c7c607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:00.859812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2fe1c-32d1-4dcf-847e-b9f538c7c607 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:00.860129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2fe1c-32d1-4dcf-847e-b9f538c7c607 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:00.860410 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2fe1c-32d1-4dcf-847e-b9f538c7c607 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:00.861030 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2fe1c-32d1-4dcf-847e-b9f538c7c607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:00.862988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2fe1c-32d1-4dcf-847e-b9f538c7c607 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.004282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 777/1581] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:00 2022] POST /identity/v3/users => generated 303 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:01.005029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2fe1c-32d1-4dcf-847e-b9f538c7c607 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.008744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 805/1582] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:00 2022] POST /identity/v3/projects => generated 377 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:01.008744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e456a-a879-401a-91f9-18018c6992a8 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.018119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2fe1c-32d1-4dcf-847e-b9f538c7c607 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.088061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e456a-a879-401a-91f9-18018c6992a8 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.090881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a53d69-2b6f-4cd1-a462-269b07b5252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.092207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a53d69-2b6f-4cd1-a462-269b07b5252b admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.092630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a53d69-2b6f-4cd1-a462-269b07b5252b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.092763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a53d69-2b6f-4cd1-a462-269b07b5252b admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.093422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a53d69-2b6f-4cd1-a462-269b07b5252b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.095052 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a53d69-2b6f-4cd1-a462-269b07b5252b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.095842 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83df52-66c5-4f1f-8452-ba1376f00e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.096506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83df52-66c5-4f1f-8452-ba1376f00e32 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.096788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83df52-66c5-4f1f-8452-ba1376f00e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.097064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83df52-66c5-4f1f-8452-ba1376f00e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.097711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83df52-66c5-4f1f-8452-ba1376f00e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.099319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83df52-66c5-4f1f-8452-ba1376f00e32 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.109948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 778/1583] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.110638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a53d69-2b6f-4cd1-a462-269b07b5252b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.111102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 806/1584] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.111977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83df52-66c5-4f1f-8452-ba1376f00e32 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.123586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83df52-66c5-4f1f-8452-ba1376f00e32 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.127371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a53d69-2b6f-4cd1-a462-269b07b5252b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.128451 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6611124-dbae-430c-82b8-cf813a956fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.129047 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6611124-dbae-430c-82b8-cf813a956fba admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.129330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6611124-dbae-430c-82b8-cf813a956fba admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.129604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6611124-dbae-430c-82b8-cf813a956fba admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.130224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6611124-dbae-430c-82b8-cf813a956fba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.132041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd711e3-e9d8-483a-9f8f-9b272d81a0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.132247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6611124-dbae-430c-82b8-cf813a956fba admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.132808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd711e3-e9d8-483a-9f8f-9b272d81a0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.133160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd711e3-e9d8-483a-9f8f-9b272d81a0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.133515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd711e3-e9d8-483a-9f8f-9b272d81a0f0 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.134381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd711e3-e9d8-483a-9f8f-9b272d81a0f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.136515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd711e3-e9d8-483a-9f8f-9b272d81a0f0 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.260810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 807/1585] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:01 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:01.261250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 779/1586] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:01 2022] POST /identity/v3/users => generated 297 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:01.262035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6611124-dbae-430c-82b8-cf813a956fba admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.263465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd711e3-e9d8-483a-9f8f-9b272d81a0f0 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.275180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd711e3-e9d8-483a-9f8f-9b272d81a0f0 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.275496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6611124-dbae-430c-82b8-cf813a956fba admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.281395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09f5a9-e511-44b4-b638-cebd5d7d9f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.281571 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ac0f5-df25-44ea-8f69-0c4dd89657f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.282114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ac0f5-df25-44ea-8f69-0c4dd89657f6 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.282401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ac0f5-df25-44ea-8f69-0c4dd89657f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.282680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ac0f5-df25-44ea-8f69-0c4dd89657f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.282806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09f5a9-e511-44b4-b638-cebd5d7d9f88 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.283327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ac0f5-df25-44ea-8f69-0c4dd89657f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.283517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09f5a9-e511-44b4-b638-cebd5d7d9f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.283977 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09f5a9-e511-44b4-b638-cebd5d7d9f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.284787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09f5a9-e511-44b4-b638-cebd5d7d9f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.285108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ac0f5-df25-44ea-8f69-0c4dd89657f6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.286848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09f5a9-e511-44b4-b638-cebd5d7d9f88 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.298658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 780/1587] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.299724 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09f5a9-e511-44b4-b638-cebd5d7d9f88 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.314000 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 808/1588] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.314000 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ac0f5-df25-44ea-8f69-0c4dd89657f6 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.319404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09f5a9-e511-44b4-b638-cebd5d7d9f88 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.327243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbd529-a814-4197-8f04-46fd339b4e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.328029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbd529-a814-4197-8f04-46fd339b4e45 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.328353 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbd529-a814-4197-8f04-46fd339b4e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.328569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbd529-a814-4197-8f04-46fd339b4e45 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.329258 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbd529-a814-4197-8f04-46fd339b4e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.331358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbd529-a814-4197-8f04-46fd339b4e45 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.334654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ac0f5-df25-44ea-8f69-0c4dd89657f6 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.340151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8184e1-786f-4e73-baf6-8d56f5f22ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.348261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8184e1-786f-4e73-baf6-8d56f5f22ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.348261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8184e1-786f-4e73-baf6-8d56f5f22ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.348261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8184e1-786f-4e73-baf6-8d56f5f22ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.348261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8184e1-786f-4e73-baf6-8d56f5f22ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.348261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8184e1-786f-4e73-baf6-8d56f5f22ae4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.356485 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 809/1589] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.357127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8184e1-786f-4e73-baf6-8d56f5f22ae4 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.367006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8184e1-786f-4e73-baf6-8d56f5f22ae4 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.371893 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d18d6-a1b8-4a94-9f2b-793f9ff4365c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.372543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d18d6-a1b8-4a94-9f2b-793f9ff4365c admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.372840 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d18d6-a1b8-4a94-9f2b-793f9ff4365c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.373109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d18d6-a1b8-4a94-9f2b-793f9ff4365c admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.373758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d18d6-a1b8-4a94-9f2b-793f9ff4365c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.375346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d18d6-a1b8-4a94-9f2b-793f9ff4365c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.385761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 810/1590] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.386558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d18d6-a1b8-4a94-9f2b-793f9ff4365c admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.396601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d18d6-a1b8-4a94-9f2b-793f9ff4365c admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.401386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4446ed5-223c-4ff9-a170-8fb289912671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.401957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4446ed5-223c-4ff9-a170-8fb289912671 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.402241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4446ed5-223c-4ff9-a170-8fb289912671 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.402518 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4446ed5-223c-4ff9-a170-8fb289912671 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.403144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4446ed5-223c-4ff9-a170-8fb289912671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.405333 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4446ed5-223c-4ff9-a170-8fb289912671 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.406666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:01.415649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 811/1591] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.416448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4446ed5-223c-4ff9-a170-8fb289912671 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.426409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4446ed5-223c-4ff9-a170-8fb289912671 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.431153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c941274-8c98-4d3a-8491-229af3e16693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.432000 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c941274-8c98-4d3a-8491-229af3e16693 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.432290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c941274-8c98-4d3a-8491-229af3e16693 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.432580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c941274-8c98-4d3a-8491-229af3e16693 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.433210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c941274-8c98-4d3a-8491-229af3e16693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.434752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c941274-8c98-4d3a-8491-229af3e16693 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.445372 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 812/1592] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.446117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c941274-8c98-4d3a-8491-229af3e16693 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.455847 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c941274-8c98-4d3a-8491-229af3e16693 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.462624 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70cb90-018d-4d66-af94-3273fd2a82b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.470493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 781/1593] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:01 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:01.470624 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70cb90-018d-4d66-af94-3273fd2a82b3 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.470624 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70cb90-018d-4d66-af94-3273fd2a82b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.470624 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70cb90-018d-4d66-af94-3273fd2a82b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.470883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbd529-a814-4197-8f04-46fd339b4e45 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.471006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70cb90-018d-4d66-af94-3273fd2a82b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.471006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70cb90-018d-4d66-af94-3273fd2a82b3 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.475155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbd529-a814-4197-8f04-46fd339b4e45 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.479777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 813/1594] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.479777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70cb90-018d-4d66-af94-3273fd2a82b3 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.483050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2437e44-9b43-4491-a135-d6d9ab7ff73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.483726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2437e44-9b43-4491-a135-d6d9ab7ff73a admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.484127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2437e44-9b43-4491-a135-d6d9ab7ff73a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.484411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2437e44-9b43-4491-a135-d6d9ab7ff73a admin admin] PATH_INFO: `/v3/projects/00ee3af9e5784b81a51565cd2fbd77ca/users/020bd75ebcf24fe9adbc1072603020bc/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.490562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70cb90-018d-4d66-af94-3273fd2a82b3 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.491606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:01.496078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec99f11-5350-4d59-8df6-4ee17ecd6fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.500552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec99f11-5350-4d59-8df6-4ee17ecd6fe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.500552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec99f11-5350-4d59-8df6-4ee17ecd6fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.500552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec99f11-5350-4d59-8df6-4ee17ecd6fe1 admin admin] PATH_INFO: `/v3/projects/d59842f7ab92458282236e743cf4b436/users/90d2721f1c1f48d28078a2d3c338fcd5/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.505757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2437e44-9b43-4491-a135-d6d9ab7ff73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=00ee3af9e5784b81a51565cd2fbd77ca, user_id=020bd75ebcf24fe9adbc1072603020bc, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.508599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2437e44-9b43-4491-a135-d6d9ab7ff73a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.514800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec99f11-5350-4d59-8df6-4ee17ecd6fe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d59842f7ab92458282236e743cf4b436, user_id=90d2721f1c1f48d28078a2d3c338fcd5, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.515822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: WARNING py.warnings [None req-3ec99f11-5350-4d59-8df6-4ee17ecd6fe1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:50:01.515822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: warnings.warn(msg) Aug 26 15:50:01.515822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: Aug 26 15:50:01.517880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec99f11-5350-4d59-8df6-4ee17ecd6fe1 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.594339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 782/1595] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:01 2022] PUT /identity/v3/projects/00ee3af9e5784b81a51565cd2fbd77ca/users/020bd75ebcf24fe9adbc1072603020bc/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:01.595862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2437e44-9b43-4491-a135-d6d9ab7ff73a admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.668687 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 814/1596] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:01 2022] PUT /identity/v3/projects/d59842f7ab92458282236e743cf4b436/users/90d2721f1c1f48d28078a2d3c338fcd5/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:01.670393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec99f11-5350-4d59-8df6-4ee17ecd6fe1 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.685422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2437e44-9b43-4491-a135-d6d9ab7ff73a admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.744370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec99f11-5350-4d59-8df6-4ee17ecd6fe1 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.749019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bac371-75a6-4af1-bf02-610cab431900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.749773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bac371-75a6-4af1-bf02-610cab431900 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.749894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcc96b-0942-4cc2-8842-7f22e698ee59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.750345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bac371-75a6-4af1-bf02-610cab431900 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.750471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcc96b-0942-4cc2-8842-7f22e698ee59 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.750682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcc96b-0942-4cc2-8842-7f22e698ee59 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.750795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bac371-75a6-4af1-bf02-610cab431900 admin admin] PATH_INFO: `/v3/projects/6198f7e071e54f0f8c5186f5b031a7b4/users/af36ee65e4c54dc38d9f876e093b1c2f/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.750966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcc96b-0942-4cc2-8842-7f22e698ee59 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.751623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcc96b-0942-4cc2-8842-7f22e698ee59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.753281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcc96b-0942-4cc2-8842-7f22e698ee59 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.766297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 815/1597] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.766890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcc96b-0942-4cc2-8842-7f22e698ee59 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.769721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bac371-75a6-4af1-bf02-610cab431900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6198f7e071e54f0f8c5186f5b031a7b4, user_id=af36ee65e4c54dc38d9f876e093b1c2f, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.772837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bac371-75a6-4af1-bf02-610cab431900 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.777026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcc96b-0942-4cc2-8842-7f22e698ee59 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.781822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975b668-4dc6-47ba-a05d-8ddb5bcd1040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.782429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975b668-4dc6-47ba-a05d-8ddb5bcd1040 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.782719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975b668-4dc6-47ba-a05d-8ddb5bcd1040 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.783004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975b668-4dc6-47ba-a05d-8ddb5bcd1040 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.783652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975b668-4dc6-47ba-a05d-8ddb5bcd1040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.785292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975b668-4dc6-47ba-a05d-8ddb5bcd1040 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.795987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 816/1598] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.796820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975b668-4dc6-47ba-a05d-8ddb5bcd1040 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.806648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975b668-4dc6-47ba-a05d-8ddb5bcd1040 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.812223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80293ec-c888-404f-a327-4446fe7e4c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.812839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80293ec-c888-404f-a327-4446fe7e4c61 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.813126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80293ec-c888-404f-a327-4446fe7e4c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.813409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80293ec-c888-404f-a327-4446fe7e4c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.814058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80293ec-c888-404f-a327-4446fe7e4c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.815671 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80293ec-c888-404f-a327-4446fe7e4c61 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.828647 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 817/1599] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:01.829513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80293ec-c888-404f-a327-4446fe7e4c61 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:01.839429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80293ec-c888-404f-a327-4446fe7e4c61 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:01.844225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77869333-851d-4162-9c0a-a24e6115ad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:01.844875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77869333-851d-4162-9c0a-a24e6115ad47 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.845163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77869333-851d-4162-9c0a-a24e6115ad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.845449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77869333-851d-4162-9c0a-a24e6115ad47 admin admin] PATH_INFO: `/v3/projects/b0b3bbc1252e45b997ec2a567d490018/users/c9ddd55f07df4a78a9b24fdf46ad9e10/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.860633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77869333-851d-4162-9c0a-a24e6115ad47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b3bbc1252e45b997ec2a567d490018, user_id=c9ddd55f07df4a78a9b24fdf46ad9e10, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:01.863422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77869333-851d-4162-9c0a-a24e6115ad47 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:01.880761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 783/1600] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:01 2022] PUT /identity/v3/projects/6198f7e071e54f0f8c5186f5b031a7b4/users/af36ee65e4c54dc38d9f876e093b1c2f/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:01.882241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395aee-cef7-448f-b970-12219f9deffb None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.882596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395aee-cef7-448f-b970-12219f9deffb None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.882938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395aee-cef7-448f-b970-12219f9deffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.909844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-83395aee-cef7-448f-b970-12219f9deffb None None] MFA Rules not processed for user `020bd75ebcf24fe9adbc1072603020bc`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:01.911054 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-83395aee-cef7-448f-b970-12219f9deffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:01.974852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 784/1601] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:01.976279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c335d7-f35d-44df-9cf4-9085fdc260e2 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:01.976569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c335d7-f35d-44df-9cf4-9085fdc260e2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:01.976846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c335d7-f35d-44df-9cf4-9085fdc260e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:01.986530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:01.996777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 818/1602] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:01 2022] PUT /identity/v3/projects/b0b3bbc1252e45b997ec2a567d490018/users/c9ddd55f07df4a78a9b24fdf46ad9e10/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:01.998813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77869333-851d-4162-9c0a-a24e6115ad47 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:02.031506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-21c335d7-f35d-44df-9cf4-9085fdc260e2 None None] MFA Rules not processed for user `90d2721f1c1f48d28078a2d3c338fcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:02.031506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-21c335d7-f35d-44df-9cf4-9085fdc260e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:02.085990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77869333-851d-4162-9c0a-a24e6115ad47 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:02.090597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9ec6e-0ec5-459a-a1f8-682e8a0726e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:02.091298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9ec6e-0ec5-459a-a1f8-682e8a0726e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.091583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9ec6e-0ec5-459a-a1f8-682e8a0726e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.091861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9ec6e-0ec5-459a-a1f8-682e8a0726e9 admin admin] PATH_INFO: `/v3/projects/6110b6959363437c91a112be1b52b230/users/0072d0b0c39a4c2384f5c6e08c1303b2/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.094104 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:02.102722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 785/1603] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:02.103558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c335d7-f35d-44df-9cf4-9085fdc260e2 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:02.108195 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9ec6e-0ec5-459a-a1f8-682e8a0726e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6110b6959363437c91a112be1b52b230, user_id=0072d0b0c39a4c2384f5c6e08c1303b2, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:02.110860 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9ec6e-0ec5-459a-a1f8-682e8a0726e9 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:02.116608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c335d7-f35d-44df-9cf4-9085fdc260e2 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:02.122187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274088b-f252-44aa-bda2-426fb7a31e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:02.123535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274088b-f252-44aa-bda2-426fb7a31e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.123824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274088b-f252-44aa-bda2-426fb7a31e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.124215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274088b-f252-44aa-bda2-426fb7a31e86 admin admin] PATH_INFO: `/v3/projects/853df33daae14bd9825788afd555e11a/users/5c01a893d3ab42cc97508768791ff38e/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.139890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274088b-f252-44aa-bda2-426fb7a31e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=853df33daae14bd9825788afd555e11a, user_id=5c01a893d3ab42cc97508768791ff38e, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:02.142650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274088b-f252-44aa-bda2-426fb7a31e86 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:02.257414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 786/1604] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:02 2022] PUT /identity/v3/projects/853df33daae14bd9825788afd555e11a/users/5c01a893d3ab42cc97508768791ff38e/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:02.257879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 819/1605] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:01 2022] PUT /identity/v3/projects/6110b6959363437c91a112be1b52b230/users/0072d0b0c39a4c2384f5c6e08c1303b2/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:02.258264 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274088b-f252-44aa-bda2-426fb7a31e86 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:02.259462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88537bbd-9931-4420-a693-51531f70ab57 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.259917 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88537bbd-9931-4420-a693-51531f70ab57 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.260203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88537bbd-9931-4420-a693-51531f70ab57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.298866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-88537bbd-9931-4420-a693-51531f70ab57 None None] MFA Rules not processed for user `af36ee65e4c54dc38d9f876e093b1c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:02.300454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-88537bbd-9931-4420-a693-51531f70ab57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:02.343275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274088b-f252-44aa-bda2-426fb7a31e86 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:02.391635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 820/1606] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:02.392564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88537bbd-9931-4420-a693-51531f70ab57 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:02.432423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9f321-d085-4136-badb-5b1d9ae981b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:02.433048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9f321-d085-4136-badb-5b1d9ae981b2 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.433334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9f321-d085-4136-badb-5b1d9ae981b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.433615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9f321-d085-4136-badb-5b1d9ae981b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.440143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9f321-d085-4136-badb-5b1d9ae981b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:02.441897 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9f321-d085-4136-badb-5b1d9ae981b2 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:02.461770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 787/1607] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:02.464579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f797cd-6646-4eb0-b861-e72725c275b2 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.464579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f797cd-6646-4eb0-b861-e72725c275b2 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.464579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f797cd-6646-4eb0-b861-e72725c275b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.466291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88537bbd-9931-4420-a693-51531f70ab57 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:02.472273 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ee3ac-293a-41ef-8b8e-93a37c8a6afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:02.472974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ee3ac-293a-41ef-8b8e-93a37c8a6afd service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.473644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ee3ac-293a-41ef-8b8e-93a37c8a6afd service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.473929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ee3ac-293a-41ef-8b8e-93a37c8a6afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.494153 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:02.497264 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-33f797cd-6646-4eb0-b861-e72725c275b2 None None] MFA Rules not processed for user `c9ddd55f07df4a78a9b24fdf46ad9e10`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:02.499347 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-33f797cd-6646-4eb0-b861-e72725c275b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:02.536608 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ee3ac-293a-41ef-8b8e-93a37c8a6afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:02.538191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ee3ac-293a-41ef-8b8e-93a37c8a6afd service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:02.548020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 821/1608] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:02.549293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45cd62-6c7f-485d-9387-f0d20e63dab6 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.549585 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45cd62-6c7f-485d-9387-f0d20e63dab6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.549866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45cd62-6c7f-485d-9387-f0d20e63dab6 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.551121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 822/1609] 10.222.0.98 () {54 vars in 962 bytes} [Fri Aug 26 15:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:02.553227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafbd77-7082-4600-901b-ac67ab0ae5ef None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.553798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafbd77-7082-4600-901b-ac67ab0ae5ef None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.554189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafbd77-7082-4600-901b-ac67ab0ae5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.582130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 788/1610] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:02 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:02.583054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0eafbd77-7082-4600-901b-ac67ab0ae5ef None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:02.583667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfd382-91dd-4b56-96d0-acadc2bceb87 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.583996 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfd382-91dd-4b56-96d0-acadc2bceb87 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.584287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfd382-91dd-4b56-96d0-acadc2bceb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.584473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0eafbd77-7082-4600-901b-ac67ab0ae5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:02.599720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 823/1611] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:02.599720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafbd77-7082-4600-901b-ac67ab0ae5ef None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:02.620545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafbd77-7082-4600-901b-ac67ab0ae5ef None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:02.628327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-07dfd382-91dd-4b56-96d0-acadc2bceb87 None None] MFA Rules not processed for user `0072d0b0c39a4c2384f5c6e08c1303b2`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:02.628327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-07dfd382-91dd-4b56-96d0-acadc2bceb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:02.628565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05c330-591f-4454-80bb-e21c014827d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:02.628565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05c330-591f-4454-80bb-e21c014827d6 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.628565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05c330-591f-4454-80bb-e21c014827d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.628565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05c330-591f-4454-80bb-e21c014827d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.629446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05c330-591f-4454-80bb-e21c014827d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:02.631717 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05c330-591f-4454-80bb-e21c014827d6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:02.720322 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 789/1612] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:02.722274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfd382-91dd-4b56-96d0-acadc2bceb87 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:02.739414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfd382-91dd-4b56-96d0-acadc2bceb87 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:02.745478 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:02.745947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dce07-f73b-4e4d-87f9-e19aa7da6df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:02.748829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dce07-f73b-4e4d-87f9-e19aa7da6df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.755038 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dce07-f73b-4e4d-87f9-e19aa7da6df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.755038 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dce07-f73b-4e4d-87f9-e19aa7da6df6 admin admin] PATH_INFO: `/v3/projects/35ef4da23ec44ed2a75e20f8cd365691/users/72dc0c14d61e4bae9aaa31bf46cb1e08/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.760870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 824/1613] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:02 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:02.760870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b519b-ceb0-4c30-b9c1-ba723f74d9d3 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.760870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b519b-ceb0-4c30-b9c1-ba723f74d9d3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.760870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b519b-ceb0-4c30-b9c1-ba723f74d9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:02.772510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dce07-f73b-4e4d-87f9-e19aa7da6df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ef4da23ec44ed2a75e20f8cd365691, user_id=72dc0c14d61e4bae9aaa31bf46cb1e08, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:02.775270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dce07-f73b-4e4d-87f9-e19aa7da6df6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:02.814321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-7d5b519b-ceb0-4c30-b9c1-ba723f74d9d3 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:02.817703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-7d5b519b-ceb0-4c30-b9c1-ba723f74d9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:02.887432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 825/1614] 10.222.0.98 () {58 vars in 1059 bytes} [Fri Aug 26 15:50:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:02.888878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b519b-ceb0-4c30-b9c1-ba723f74d9d3 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:02.901819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 790/1615] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:02 2022] PUT /identity/v3/projects/35ef4da23ec44ed2a75e20f8cd365691/users/72dc0c14d61e4bae9aaa31bf46cb1e08/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:02.902831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dce07-f73b-4e4d-87f9-e19aa7da6df6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:02.906396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-084dce07-f73b-4e4d-87f9-e19aa7da6df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:02.977159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dce07-f73b-4e4d-87f9-e19aa7da6df6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:02.978036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b519b-ceb0-4c30-b9c1-ba723f74d9d3 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:02.979545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc99f0-7dd2-450c-bf79-d34302c1481e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:02.980977 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc99f0-7dd2-450c-bf79-d34302c1481e service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:02.981371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc99f0-7dd2-450c-bf79-d34302c1481e service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:02.981860 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc99f0-7dd2-450c-bf79-d34302c1481e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.050159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2567e-88bb-44d7-aa17-dcfc956d1f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:03.050159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2567e-88bb-44d7-aa17-dcfc956d1f11 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.050159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2567e-88bb-44d7-aa17-dcfc956d1f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.050159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2567e-88bb-44d7-aa17-dcfc956d1f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.050788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2567e-88bb-44d7-aa17-dcfc956d1f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:03.050788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2567e-88bb-44d7-aa17-dcfc956d1f11 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:03.052397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc99f0-7dd2-450c-bf79-d34302c1481e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:03.053920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc99f0-7dd2-450c-bf79-d34302c1481e service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:03.064798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 791/1616] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:03.066227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc15b8-7cdc-4d2d-a6cb-c4182f359b4c None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.066573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc15b8-7cdc-4d2d-a6cb-c4182f359b4c None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.066855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc15b8-7cdc-4d2d-a6cb-c4182f359b4c None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.068413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 792/1617] 10.222.0.98 () {54 vars in 962 bytes} [Fri Aug 26 15:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:03.069115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc15b8-7cdc-4d2d-a6cb-c4182f359b4c None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:03.085083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc15b8-7cdc-4d2d-a6cb-c4182f359b4c None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:03.091900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55436d37-4b8c-4562-9c64-3242ff415c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:03.092573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55436d37-4b8c-4562-9c64-3242ff415c37 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.092886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55436d37-4b8c-4562-9c64-3242ff415c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.093161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55436d37-4b8c-4562-9c64-3242ff415c37 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.093826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55436d37-4b8c-4562-9c64-3242ff415c37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:03.095739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55436d37-4b8c-4562-9c64-3242ff415c37 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:03.261777 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 793/1618] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:03 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:03.262001 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 826/1619] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:02 2022] POST /identity/v3/projects => generated 403 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:03.262450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55436d37-4b8c-4562-9c64-3242ff415c37 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:03.262568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9449-489b-42fd-a2db-16db3a2f8147 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.262568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9449-489b-42fd-a2db-16db3a2f8147 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.262568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9449-489b-42fd-a2db-16db3a2f8147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.290338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1f6f9449-489b-42fd-a2db-16db3a2f8147 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:03.291594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1f6f9449-489b-42fd-a2db-16db3a2f8147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:03.379284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55436d37-4b8c-4562-9c64-3242ff415c37 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:03.399935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 827/1620] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:03.402371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ecd8b-506d-45c2-9803-49eaed819980 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.402912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ecd8b-506d-45c2-9803-49eaed819980 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.403688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ecd8b-506d-45c2-9803-49eaed819980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.437801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-760ecd8b-506d-45c2-9803-49eaed819980 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:03.440113 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-760ecd8b-506d-45c2-9803-49eaed819980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:03.491635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58584fbd-db67-41e4-b100-2de282f6c6c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:03.492583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58584fbd-db67-41e4-b100-2de282f6c6c3 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.492878 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58584fbd-db67-41e4-b100-2de282f6c6c3 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.493159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58584fbd-db67-41e4-b100-2de282f6c6c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.532806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 828/1621] 10.222.0.98 () {58 vars in 1059 bytes} [Fri Aug 26 15:50:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:03.550253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ecd8b-506d-45c2-9803-49eaed819980 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:03.576524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ecd8b-506d-45c2-9803-49eaed819980 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:03.594208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a4ae6-2a07-4d20-be67-00af59c35bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:03.788552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a4ae6-2a07-4d20-be67-00af59c35bcc admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.788552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a4ae6-2a07-4d20-be67-00af59c35bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.788552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a4ae6-2a07-4d20-be67-00af59c35bcc admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.794989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58584fbd-db67-41e4-b100-2de282f6c6c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:03.794989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58584fbd-db67-41e4-b100-2de282f6c6c3 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:03.798281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a4ae6-2a07-4d20-be67-00af59c35bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:03.798281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a4ae6-2a07-4d20-be67-00af59c35bcc admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:03.798674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 794/1622] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:03.798674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58584fbd-db67-41e4-b100-2de282f6c6c3 service nova] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:03.798674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58584fbd-db67-41e4-b100-2de282f6c6c3 service nova] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:03.798674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b38eba-2d1c-4ce2-980f-986c57a1b4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:03.799206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b38eba-2d1c-4ce2-980f-986c57a1b4ae admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.799206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b38eba-2d1c-4ce2-980f-986c57a1b4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.799206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b38eba-2d1c-4ce2-980f-986c57a1b4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.799206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b38eba-2d1c-4ce2-980f-986c57a1b4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:03.799206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b38eba-2d1c-4ce2-980f-986c57a1b4ae admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:03.799206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 795/1623] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:03.800928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b38eba-2d1c-4ce2-980f-986c57a1b4ae admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:03.812711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 829/1624] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:03 2022] POST /identity/v3/users => generated 308 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:03.813531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a4ae6-2a07-4d20-be67-00af59c35bcc admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:03.849931 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b38eba-2d1c-4ce2-980f-986c57a1b4ae admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:03.850370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a4ae6-2a07-4d20-be67-00af59c35bcc admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:03.860236 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cdab58-360b-472c-9a07-d47661f4c56c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:03.860870 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdab58-360b-472c-9a07-d47661f4c56c service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.861154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdab58-360b-472c-9a07-d47661f4c56c service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.861429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdab58-360b-472c-9a07-d47661f4c56c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.863633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dd358-ccd5-424d-8d5e-d2396fe2dc7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:03.864404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd358-ccd5-424d-8d5e-d2396fe2dc7d service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.864799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd358-ccd5-424d-8d5e-d2396fe2dc7d service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.865192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd358-ccd5-424d-8d5e-d2396fe2dc7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.872966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cdab58-360b-472c-9a07-d47661f4c56c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:03.874576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cdab58-360b-472c-9a07-d47661f4c56c service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:03.886250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 796/1625] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:03.887188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cdab58-360b-472c-9a07-d47661f4c56c service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:03.898246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cdab58-360b-472c-9a07-d47661f4c56c service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:03.903664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:03.905519 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f17eba-f269-4a9e-9017-3939fa6a3f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:03.906297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f17eba-f269-4a9e-9017-3939fa6a3f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.907398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f17eba-f269-4a9e-9017-3939fa6a3f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.907689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f17eba-f269-4a9e-9017-3939fa6a3f69 admin admin] PATH_INFO: `/v3/projects/3b775594683d4bf197115b2dfa339c89/users/a75382812ffb48a892013e24eb5117d2/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.924261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f17eba-f269-4a9e-9017-3939fa6a3f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b775594683d4bf197115b2dfa339c89, user_id=a75382812ffb48a892013e24eb5117d2, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:03.927044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f17eba-f269-4a9e-9017-3939fa6a3f69 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:03.947633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dd358-ccd5-424d-8d5e-d2396fe2dc7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:03.949596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dd358-ccd5-424d-8d5e-d2396fe2dc7d service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:03.961121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 830/1626] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:03.962119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dd358-ccd5-424d-8d5e-d2396fe2dc7d service nova] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:03.973163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dd358-ccd5-424d-8d5e-d2396fe2dc7d service nova] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:03.978252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e9196-cf09-4c5e-a754-c82b1eaec295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:03.979012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e9196-cf09-4c5e-a754-c82b1eaec295 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:03.979299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e9196-cf09-4c5e-a754-c82b1eaec295 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:03.979581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e9196-cf09-4c5e-a754-c82b1eaec295 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:03.980330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e9196-cf09-4c5e-a754-c82b1eaec295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:03.982137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e9196-cf09-4c5e-a754-c82b1eaec295 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:04.011255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 831/1627] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:04.012878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e9196-cf09-4c5e-a754-c82b1eaec295 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:04.042929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e9196-cf09-4c5e-a754-c82b1eaec295 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:04.048785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6021469-c258-48f3-9222-e694819bf585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:04.049566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6021469-c258-48f3-9222-e694819bf585 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:04.049955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6021469-c258-48f3-9222-e694819bf585 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:04.050235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6021469-c258-48f3-9222-e694819bf585 admin admin] PATH_INFO: `/v3/projects/627a9090bf0f465ea9393eee90193b13/users/ff2377363d8c4a279531455bb4b8f84a/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:04.085116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6021469-c258-48f3-9222-e694819bf585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=627a9090bf0f465ea9393eee90193b13, user_id=ff2377363d8c4a279531455bb4b8f84a, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:04.090177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6021469-c258-48f3-9222-e694819bf585 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:04.136397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 797/1628] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:03 2022] PUT /identity/v3/projects/3b775594683d4bf197115b2dfa339c89/users/a75382812ffb48a892013e24eb5117d2/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:04.145620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a997f-b9c7-4869-8c58-991d84d052be None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:04.147451 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a997f-b9c7-4869-8c58-991d84d052be None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:04.147745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a997f-b9c7-4869-8c58-991d84d052be None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:04.175924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 798/1629] 10.222.0.98 () {54 vars in 912 bytes} [Fri Aug 26 15:50:04 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:04.177157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a997f-b9c7-4869-8c58-991d84d052be None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:04.243259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:04.256647 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 832/1630] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:04 2022] PUT /identity/v3/projects/627a9090bf0f465ea9393eee90193b13/users/ff2377363d8c4a279531455bb4b8f84a/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:04.258029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeebcb6-6ba1-4b2c-8297-b1a41a605d98 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:04.258324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeebcb6-6ba1-4b2c-8297-b1a41a605d98 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:04.258606 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeebcb6-6ba1-4b2c-8297-b1a41a605d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:04.302325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-cdeebcb6-6ba1-4b2c-8297-b1a41a605d98 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:04.303683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cdeebcb6-6ba1-4b2c-8297-b1a41a605d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:04.384162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a997f-b9c7-4869-8c58-991d84d052be None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:04.472339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 833/1631] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:50:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:04.473315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeebcb6-6ba1-4b2c-8297-b1a41a605d98 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:04.519835 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aaa9b-5289-48f8-8557-2848838f076a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:04.520798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aaa9b-5289-48f8-8557-2848838f076a admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:04.521404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aaa9b-5289-48f8-8557-2848838f076a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:04.521872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aaa9b-5289-48f8-8557-2848838f076a admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:04.523693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aaa9b-5289-48f8-8557-2848838f076a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:04.549691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aaa9b-5289-48f8-8557-2848838f076a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:04.590356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 799/1632] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:04.592365 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aaa9b-5289-48f8-8557-2848838f076a admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:04.602406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeebcb6-6ba1-4b2c-8297-b1a41a605d98 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:04.628977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6139a2a-fd02-4283-9a93-b257ca4256c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:04.635182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6139a2a-fd02-4283-9a93-b257ca4256c1 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:04.635182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6139a2a-fd02-4283-9a93-b257ca4256c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:04.635182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6139a2a-fd02-4283-9a93-b257ca4256c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:04.635182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6139a2a-fd02-4283-9a93-b257ca4256c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:04.642351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6139a2a-fd02-4283-9a93-b257ca4256c1 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:04.677892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 834/1633] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:04.679357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0e743-4732-4ab4-bc30-d76e20f7d319 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:04.679626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0e743-4732-4ab4-bc30-d76e20f7d319 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:04.679929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0e743-4732-4ab4-bc30-d76e20f7d319 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:04.686296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 835/1634] 10.222.0.98 () {54 vars in 912 bytes} [Fri Aug 26 15:50:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:04.688633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0e743-4732-4ab4-bc30-d76e20f7d319 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:04.720614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aaa9b-5289-48f8-8557-2848838f076a admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:04.725957 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ca502-bda5-4240-b260-1b15c2b5d298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:04.726158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0e743-4732-4ab4-bc30-d76e20f7d319 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:04.727238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ca502-bda5-4240-b260-1b15c2b5d298 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:04.728061 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ca502-bda5-4240-b260-1b15c2b5d298 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:04.728504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ca502-bda5-4240-b260-1b15c2b5d298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:04.737518 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1341ae-5935-4628-8ac4-b321876ca8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:04.738032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1341ae-5935-4628-8ac4-b321876ca8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:04.740835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1341ae-5935-4628-8ac4-b321876ca8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:04.741188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1341ae-5935-4628-8ac4-b321876ca8bf admin admin] PATH_INFO: `/v3/projects/3b775594683d4bf197115b2dfa339c89/users/a75382812ffb48a892013e24eb5117d2/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:04.747269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1341ae-5935-4628-8ac4-b321876ca8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b775594683d4bf197115b2dfa339c89, user_id=a75382812ffb48a892013e24eb5117d2, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:04.751551 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ca502-bda5-4240-b260-1b15c2b5d298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:04.751677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1341ae-5935-4628-8ac4-b321876ca8bf admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:04.762024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ca502-bda5-4240-b260-1b15c2b5d298 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:04.779234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 800/1635] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:04.780229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ca502-bda5-4240-b260-1b15c2b5d298 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:04.791224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ca502-bda5-4240-b260-1b15c2b5d298 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:04.800367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 836/1636] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:04 2022] PUT /identity/v3/projects/3b775594683d4bf197115b2dfa339c89/users/a75382812ffb48a892013e24eb5117d2/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:04.800367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b438702-e0ac-4149-9d3e-6b7882950ac5 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:04.800367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b438702-e0ac-4149-9d3e-6b7882950ac5 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:04.800367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b438702-e0ac-4149-9d3e-6b7882950ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:04.870504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-4b438702-e0ac-4149-9d3e-6b7882950ac5 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:04.871499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625a0d1-5d38-4dba-8ef3-b46e6846d07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:04.871759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4b438702-e0ac-4149-9d3e-6b7882950ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:04.872437 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625a0d1-5d38-4dba-8ef3-b46e6846d07f admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:04.873393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625a0d1-5d38-4dba-8ef3-b46e6846d07f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:04.873955 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625a0d1-5d38-4dba-8ef3-b46e6846d07f admin admin] PATH_INFO: `/v3/projects/627a9090bf0f465ea9393eee90193b13/users/ff2377363d8c4a279531455bb4b8f84a/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:04.876429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625a0d1-5d38-4dba-8ef3-b46e6846d07f admin admin] RBAC: Authorizing `identity:create_grant(project_id=627a9090bf0f465ea9393eee90193b13, user_id=ff2377363d8c4a279531455bb4b8f84a, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:04.879930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625a0d1-5d38-4dba-8ef3-b46e6846d07f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:04.903939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 801/1637] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:04 2022] PUT /identity/v3/projects/627a9090bf0f465ea9393eee90193b13/users/ff2377363d8c4a279531455bb4b8f84a/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:04.903939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daf0df-7c14-4070-9e05-9bc10467fbd0 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:04.903939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daf0df-7c14-4070-9e05-9bc10467fbd0 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:04.903939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daf0df-7c14-4070-9e05-9bc10467fbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:04.942132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b4daf0df-7c14-4070-9e05-9bc10467fbd0 None None] MFA Rules not processed for user `a75382812ffb48a892013e24eb5117d2`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:04.943770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b4daf0df-7c14-4070-9e05-9bc10467fbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:05.300989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:05.300989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 802/1638] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:04 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:05.300989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39224b7-ed69-4241-a2b3-aa87055d9c12 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:05.300989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39224b7-ed69-4241-a2b3-aa87055d9c12 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:05.303092 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 837/1639] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:50:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:05.304110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39224b7-ed69-4241-a2b3-aa87055d9c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:05.304110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-d39224b7-ed69-4241-a2b3-aa87055d9c12 None None] MFA Rules not processed for user `ff2377363d8c4a279531455bb4b8f84a`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:05.304110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d39224b7-ed69-4241-a2b3-aa87055d9c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:05.304402 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b438702-e0ac-4149-9d3e-6b7882950ac5 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:05.307168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 803/1640] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:05 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:05.307444 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b438702-e0ac-4149-9d3e-6b7882950ac5 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:05.351895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80161344-3b79-4976-8798-04a3ed113754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:05.351895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80161344-3b79-4976-8798-04a3ed113754 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:05.351895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80161344-3b79-4976-8798-04a3ed113754 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:05.351895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80161344-3b79-4976-8798-04a3ed113754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:05.362319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80161344-3b79-4976-8798-04a3ed113754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:05.364828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80161344-3b79-4976-8798-04a3ed113754 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:05.389038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 838/1641] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:50:05.544261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39224b7-ed69-4241-a2b3-aa87055d9c12 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:05.555338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39224b7-ed69-4241-a2b3-aa87055d9c12 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:05.559321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3a059-9acb-42bc-9c71-84a18e169c88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:05.560488 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3a059-9acb-42bc-9c71-84a18e169c88 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:05.560886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3a059-9acb-42bc-9c71-84a18e169c88 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:05.561162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3a059-9acb-42bc-9c71-84a18e169c88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:05.660710 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3a059-9acb-42bc-9c71-84a18e169c88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:05.662382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3a059-9acb-42bc-9c71-84a18e169c88 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:05.677476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 804/1642] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:05.714615 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:06.004631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80161344-3b79-4976-8798-04a3ed113754 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:06.016603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80161344-3b79-4976-8798-04a3ed113754 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:06.031975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1589dbb-59fc-449f-b682-855d6348b94c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:06.032675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1589dbb-59fc-449f-b682-855d6348b94c service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:06.033034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1589dbb-59fc-449f-b682-855d6348b94c service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:06.033383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1589dbb-59fc-449f-b682-855d6348b94c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:06.043980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1589dbb-59fc-449f-b682-855d6348b94c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:06.046305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1589dbb-59fc-449f-b682-855d6348b94c service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:06.063672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 839/1643] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:06 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:06.733399 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3a059-9acb-42bc-9c71-84a18e169c88 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:06.771720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3a059-9acb-42bc-9c71-84a18e169c88 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:06.780111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c59f8f-ba00-4230-a942-4755048dea96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:06.782504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59f8f-ba00-4230-a942-4755048dea96 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:06.783246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59f8f-ba00-4230-a942-4755048dea96 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:06.783907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59f8f-ba00-4230-a942-4755048dea96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:06.879595 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c59f8f-ba00-4230-a942-4755048dea96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:06.883586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c59f8f-ba00-4230-a942-4755048dea96 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:06.911122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 805/1644] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:07.203540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:07.317733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1589dbb-59fc-449f-b682-855d6348b94c service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:07.476881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1589dbb-59fc-449f-b682-855d6348b94c service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:07.494311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:07.496225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:07.496530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:07.496867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] PATH_INFO: `/v3/users/c9ddd55f07df4a78a9b24fdf46ad9e10` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:07.501149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9ddd55f07df4a78a9b24fdf46ad9e10)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:07.503668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: WARNING py.warnings [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:50:07.503668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: warnings.warn(msg) Aug 26 15:50:07.503668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: Aug 26 15:50:07.507632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:07.832341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:07.866597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9ddd55f07df4a78a9b24fdf46ad9e10'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:07.912982 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9ddd55f07df4a78a9b24fdf46ad9e10'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:07.913727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9ddd55f07df4a78a9b24fdf46ad9e10'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:07.915294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9ddd55f07df4a78a9b24fdf46ad9e10'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:08.016903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9ddd55f07df4a78a9b24fdf46ad9e10'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:08.080686 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 840/1645] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:07 2022] DELETE /identity/v3/users/c9ddd55f07df4a78a9b24fdf46ad9e10 => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:08.085650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cc886-fccd-417e-a1b1-f849b0244572 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:08.093850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cc886-fccd-417e-a1b1-f849b0244572 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:08.093850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cc886-fccd-417e-a1b1-f849b0244572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:08.115488 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-2b9cc886-fccd-417e-a1b1-f849b0244572 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:08.116920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-2b9cc886-fccd-417e-a1b1-f849b0244572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:08.265266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 806/1646] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:08.276193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:08.277955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:08.527598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8b07c-12e9-4ba7-8212-084d8ed232a3 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:08.536218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaaf1b-546a-4d6c-a27d-48948e7af0bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:08.537284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaaf1b-546a-4d6c-a27d-48948e7af0bc service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:08.539066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaaf1b-546a-4d6c-a27d-48948e7af0bc service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:08.539490 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaaf1b-546a-4d6c-a27d-48948e7af0bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:08.597827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becaaf1b-546a-4d6c-a27d-48948e7af0bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:08.616140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becaaf1b-546a-4d6c-a27d-48948e7af0bc service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:08.668642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 841/1647] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:50:09.501768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:10.196815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:10.373918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9cc886-fccd-417e-a1b1-f849b0244572 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:10.378627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-2b9cc886-fccd-417e-a1b1-f849b0244572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:10.396949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9cc886-fccd-417e-a1b1-f849b0244572 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:10.435921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abdf20-a0af-489e-b85b-8f445e934992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:10.436842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abdf20-a0af-489e-b85b-8f445e934992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:10.437724 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abdf20-a0af-489e-b85b-8f445e934992 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:10.438156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abdf20-a0af-489e-b85b-8f445e934992 admin admin] PATH_INFO: `/v3/users/90d2721f1c1f48d28078a2d3c338fcd5` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:10.441493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abdf20-a0af-489e-b85b-8f445e934992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90d2721f1c1f48d28078a2d3c338fcd5)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:10.441867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaaf1b-546a-4d6c-a27d-48948e7af0bc service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:10.443728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-becaaf1b-546a-4d6c-a27d-48948e7af0bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:10.444969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abdf20-a0af-489e-b85b-8f445e934992 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:10.477006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaaf1b-546a-4d6c-a27d-48948e7af0bc service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:10.498763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:10.502211 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:10.502507 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:10.502796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] PATH_INFO: `/v3/projects/b0b3bbc1252e45b997ec2a567d490018` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:10.512360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b3bbc1252e45b997ec2a567d490018)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:10.515094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:10.848119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b3bbc1252e45b997ec2a567d490018', 'request_id': 'req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:10.854275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-36abdf20-a0af-489e-b85b-8f445e934992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90d2721f1c1f48d28078a2d3c338fcd5'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:10.855291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-36abdf20-a0af-489e-b85b-8f445e934992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90d2721f1c1f48d28078a2d3c338fcd5'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:10.887645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-36abdf20-a0af-489e-b85b-8f445e934992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90d2721f1c1f48d28078a2d3c338fcd5'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:10.960230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b3bbc1252e45b997ec2a567d490018', 'request_id': 'req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:10.973976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b3bbc1252e45b997ec2a567d490018', 'request_id': 'req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:11.007949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] The token cache is being invalidate because project b0b3bbc1252e45b997ec2a567d490018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:11.007949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:11.008116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:11.037028 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-36abdf20-a0af-489e-b85b-8f445e934992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90d2721f1c1f48d28078a2d3c338fcd5'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:11.038847 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 842/1648] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:10 2022] DELETE /identity/v3/projects/b0b3bbc1252e45b997ec2a567d490018 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:11.044638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:11.049814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:11.050479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-36abdf20-a0af-489e-b85b-8f445e934992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90d2721f1c1f48d28078a2d3c338fcd5'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:11.072756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 807/1649] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:10 2022] DELETE /identity/v3/users/90d2721f1c1f48d28078a2d3c338fcd5 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:11.076879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fef0f-d000-49fd-8b2b-1a0789d004d1 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:11.079184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fef0f-d000-49fd-8b2b-1a0789d004d1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:11.088650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fef0f-d000-49fd-8b2b-1a0789d004d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:11.149782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-db5fef0f-d000-49fd-8b2b-1a0789d004d1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:11.150120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-db5fef0f-d000-49fd-8b2b-1a0789d004d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:11.230068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba631516-c9ff-4a08-9d2d-4ddf9f88f2a2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:11.290822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 808/1650] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:11.292419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a7b87-9a0f-4b29-a727-58fec850f9f3 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:11.292688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a7b87-9a0f-4b29-a727-58fec850f9f3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:11.292980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a7b87-9a0f-4b29-a727-58fec850f9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:11.335059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-026a7b87-9a0f-4b29-a727-58fec850f9f3 None None] MFA Rules not processed for user `6861433b9f4441fe842b506def255c58`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:11.337289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-026a7b87-9a0f-4b29-a727-58fec850f9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:11.356698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:11.371165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:11.371165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:11.371165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] PATH_INFO: `/v3/users/ff2377363d8c4a279531455bb4b8f84a` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:11.371165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff2377363d8c4a279531455bb4b8f84a)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:11.371165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:11.431168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 809/1651] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:11.432774 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a7b87-9a0f-4b29-a727-58fec850f9f3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:11.436208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-026a7b87-9a0f-4b29-a727-58fec850f9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:11.514425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a7b87-9a0f-4b29-a727-58fec850f9f3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:11.518811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189089f-68f7-45ef-bf97-c3a20289f8bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:11.519667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189089f-68f7-45ef-bf97-c3a20289f8bc service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:11.520201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189089f-68f7-45ef-bf97-c3a20289f8bc service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:11.520767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189089f-68f7-45ef-bf97-c3a20289f8bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:11.532012 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189089f-68f7-45ef-bf97-c3a20289f8bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:11.534056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189089f-68f7-45ef-bf97-c3a20289f8bc service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:11.560493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 810/1652] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:11.561876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189089f-68f7-45ef-bf97-c3a20289f8bc service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:11.569456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e189089f-68f7-45ef-bf97-c3a20289f8bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:11.685825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff2377363d8c4a279531455bb4b8f84a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:11.709581 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:11.714268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff2377363d8c4a279531455bb4b8f84a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:11.715509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff2377363d8c4a279531455bb4b8f84a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:11.717029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff2377363d8c4a279531455bb4b8f84a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:11.787452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189089f-68f7-45ef-bf97-c3a20289f8bc service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:11.803256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb797c82-cf03-440a-b321-1560e172f0e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:11.803995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb797c82-cf03-440a-b321-1560e172f0e7 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:11.804283 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb797c82-cf03-440a-b321-1560e172f0e7 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:11.804603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb797c82-cf03-440a-b321-1560e172f0e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:11.807595 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-fb797c82-cf03-440a-b321-1560e172f0e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:11.809206 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff2377363d8c4a279531455bb4b8f84a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:11.820659 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb797c82-cf03-440a-b321-1560e172f0e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:11.822391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb797c82-cf03-440a-b321-1560e172f0e7 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:11.833291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 811/1653] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:11.834324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 843/1654] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:11 2022] DELETE /identity/v3/users/ff2377363d8c4a279531455bb4b8f84a => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:12.324503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:12.384127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb797c82-cf03-440a-b321-1560e172f0e7 service nova] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:12.387271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-fb797c82-cf03-440a-b321-1560e172f0e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:12.410367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb797c82-cf03-440a-b321-1560e172f0e7 service nova] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:12.504888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:12.505834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:12.506088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:12.506475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] PATH_INFO: `/v3/projects/d59842f7ab92458282236e743cf4b436` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:12.507779 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d59842f7ab92458282236e743cf4b436)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:12.509172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: WARNING py.warnings [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:50:12.509172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: warnings.warn(msg) Aug 26 15:50:12.509172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: Aug 26 15:50:12.511056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:12.722031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd59842f7ab92458282236e743cf4b436', 'request_id': 'req-4b7ae798-a9fd-4781-b3d0-9f28d3751049'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:12.724648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:12.726371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:12.734132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd59842f7ab92458282236e743cf4b436', 'request_id': 'req-4b7ae798-a9fd-4781-b3d0-9f28d3751049'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:12.748708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd59842f7ab92458282236e743cf4b436', 'request_id': 'req-4b7ae798-a9fd-4781-b3d0-9f28d3751049'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:12.751465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78fe8d-a264-47ef-a36f-3a88e84a3af2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:12.837762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:12.845709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] The token cache is being invalidate because project d59842f7ab92458282236e743cf4b436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:12.846011 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:12.847033 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4b7ae798-a9fd-4781-b3d0-9f28d3751049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:12.856106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 812/1655] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:12 2022] DELETE /identity/v3/projects/d59842f7ab92458282236e743cf4b436 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:12.857597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9110ff-fbbc-459c-930c-e199a47dadd3 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:12.857892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9110ff-fbbc-459c-930c-e199a47dadd3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:12.858239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9110ff-fbbc-459c-930c-e199a47dadd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:12.867746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:12.868835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:12.869271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:12.869704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] PATH_INFO: `/v3/projects/627a9090bf0f465ea9393eee90193b13` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:12.871155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=627a9090bf0f465ea9393eee90193b13)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:12.873918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:12.903924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ca9110ff-fbbc-459c-930c-e199a47dadd3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:12.906245 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ca9110ff-fbbc-459c-930c-e199a47dadd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:12.932974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 813/1656] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:12.936091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b98f92-a48d-4c7f-b149-ea8393401a65 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:12.936387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b98f92-a48d-4c7f-b149-ea8393401a65 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:12.936727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b98f92-a48d-4c7f-b149-ea8393401a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:12.966102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-a4b98f92-a48d-4c7f-b149-ea8393401a65 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:12.967475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a4b98f92-a48d-4c7f-b149-ea8393401a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:12.984109 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 814/1657] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:12.985489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b98f92-a48d-4c7f-b149-ea8393401a65 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:12.998756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b98f92-a48d-4c7f-b149-ea8393401a65 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:13.006253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070a5b4-7457-495e-8040-cb0aa3e72e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:13.011528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070a5b4-7457-495e-8040-cb0aa3e72e7a admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.011528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070a5b4-7457-495e-8040-cb0aa3e72e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.011528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070a5b4-7457-495e-8040-cb0aa3e72e7a admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.011528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070a5b4-7457-495e-8040-cb0aa3e72e7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:13.011528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070a5b4-7457-495e-8040-cb0aa3e72e7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:13.014140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070a5b4-7457-495e-8040-cb0aa3e72e7a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:13.034704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 815/1658] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:13.035673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070a5b4-7457-495e-8040-cb0aa3e72e7a admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:13.050043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070a5b4-7457-495e-8040-cb0aa3e72e7a admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:13.076647 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '627a9090bf0f465ea9393eee90193b13', 'request_id': 'req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:13.134174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82215c2-770d-4c3b-893e-c012b8f50076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:13.134654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82215c2-770d-4c3b-893e-c012b8f50076 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.134937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82215c2-770d-4c3b-893e-c012b8f50076 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.135215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82215c2-770d-4c3b-893e-c012b8f50076 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.136069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82215c2-770d-4c3b-893e-c012b8f50076 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:13.138981 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82215c2-770d-4c3b-893e-c012b8f50076 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:13.184155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '627a9090bf0f465ea9393eee90193b13', 'request_id': 'req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:13.195817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '627a9090bf0f465ea9393eee90193b13', 'request_id': 'req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:13.221306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] The token cache is being invalidate because project 627a9090bf0f465ea9393eee90193b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:13.221859 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:13.222758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9f08db1b-8f7c-469a-b50c-bea3a9e7dab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:13.228198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 844/1659] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:12 2022] DELETE /identity/v3/projects/627a9090bf0f465ea9393eee90193b13 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:50:13.233337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd82c9e-19be-4712-9c2e-70f4150d36f3 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.233764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd82c9e-19be-4712-9c2e-70f4150d36f3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.234196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd82c9e-19be-4712-9c2e-70f4150d36f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.267273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 816/1660] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:13 2022] POST /identity/v3/projects => generated 371 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:13.271368 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82215c2-770d-4c3b-893e-c012b8f50076 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:13.275295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c82215c2-770d-4c3b-893e-c012b8f50076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:13.295372 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9fd82c9e-19be-4712-9c2e-70f4150d36f3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:13.302219 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9fd82c9e-19be-4712-9c2e-70f4150d36f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:13.421295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82215c2-770d-4c3b-893e-c012b8f50076 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:13.423228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 845/1661] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:13.425158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225d07b-1bf3-4793-bae4-a3b39661037a None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.425501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225d07b-1bf3-4793-bae4-a3b39661037a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.425785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225d07b-1bf3-4793-bae4-a3b39661037a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.435236 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c619f2f-2488-44c3-aaae-53e151ba09b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:13.435903 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c619f2f-2488-44c3-aaae-53e151ba09b2 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.436178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c619f2f-2488-44c3-aaae-53e151ba09b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.436453 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c619f2f-2488-44c3-aaae-53e151ba09b2 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.437463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c619f2f-2488-44c3-aaae-53e151ba09b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:13.440455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c619f2f-2488-44c3-aaae-53e151ba09b2 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:13.452519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-b225d07b-1bf3-4793-bae4-a3b39661037a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:13.453865 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-b225d07b-1bf3-4793-bae4-a3b39661037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:13.480429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 846/1662] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:13.481471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225d07b-1bf3-4793-bae4-a3b39661037a None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:13.498363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225d07b-1bf3-4793-bae4-a3b39661037a None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:13.504742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af005bf8-4bde-44d2-9851-0f64b39def30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:13.505435 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af005bf8-4bde-44d2-9851-0f64b39def30 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.505758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af005bf8-4bde-44d2-9851-0f64b39def30 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.506094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af005bf8-4bde-44d2-9851-0f64b39def30 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.506771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af005bf8-4bde-44d2-9851-0f64b39def30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:13.507331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af005bf8-4bde-44d2-9851-0f64b39def30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:13.509597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af005bf8-4bde-44d2-9851-0f64b39def30 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:13.528046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 847/1663] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:13.531855 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af005bf8-4bde-44d2-9851-0f64b39def30 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:13.544775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af005bf8-4bde-44d2-9851-0f64b39def30 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:13.550674 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1f328-9626-49e1-935c-cac547b4ff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:13.551315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1f328-9626-49e1-935c-cac547b4ff4e admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.551575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1f328-9626-49e1-935c-cac547b4ff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.551897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1f328-9626-49e1-935c-cac547b4ff4e admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.552547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1f328-9626-49e1-935c-cac547b4ff4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:13.554406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1f328-9626-49e1-935c-cac547b4ff4e admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:13.613330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 817/1664] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:13 2022] POST /identity/v3/users => generated 286 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:13.617234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c619f2f-2488-44c3-aaae-53e151ba09b2 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:13.630014 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c619f2f-2488-44c3-aaae-53e151ba09b2 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:13.647567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fe239-b3cc-41b9-a76f-0f542220691b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:13.648510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fe239-b3cc-41b9-a76f-0f542220691b admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.648795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fe239-b3cc-41b9-a76f-0f542220691b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.649078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fe239-b3cc-41b9-a76f-0f542220691b admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.650002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884fe239-b3cc-41b9-a76f-0f542220691b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:13.652146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884fe239-b3cc-41b9-a76f-0f542220691b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:13.665825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 818/1665] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:13.678075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fe239-b3cc-41b9-a76f-0f542220691b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:13.697720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fe239-b3cc-41b9-a76f-0f542220691b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:13.721846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cb617-a0aa-4d2f-b8a2-e8e514b7e769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:13.721846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cb617-a0aa-4d2f-b8a2-e8e514b7e769 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.721846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cb617-a0aa-4d2f-b8a2-e8e514b7e769 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.721846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cb617-a0aa-4d2f-b8a2-e8e514b7e769 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.722216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cb617-a0aa-4d2f-b8a2-e8e514b7e769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:13.722216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cb617-a0aa-4d2f-b8a2-e8e514b7e769 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:13.732958 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 819/1666] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:13.733950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cb617-a0aa-4d2f-b8a2-e8e514b7e769 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:13.767593 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 848/1667] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:13 2022] POST /identity/v3/projects => generated 365 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:13.767725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cb617-a0aa-4d2f-b8a2-e8e514b7e769 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:13.790032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1f328-9626-49e1-935c-cac547b4ff4e admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:13.871043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc61681-348b-49cc-87c2-bbf7934f67fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:13.871747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc61681-348b-49cc-87c2-bbf7934f67fa admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.872052 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc61681-348b-49cc-87c2-bbf7934f67fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.872337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc61681-348b-49cc-87c2-bbf7934f67fa admin admin] PATH_INFO: `/v3/projects/286e5e2ac9014adbb153202fdb2e48aa/users/1af1b25ebf86490a979440fd0e757255/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.897631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc61681-348b-49cc-87c2-bbf7934f67fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=286e5e2ac9014adbb153202fdb2e48aa, user_id=1af1b25ebf86490a979440fd0e757255, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:13.900449 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc61681-348b-49cc-87c2-bbf7934f67fa admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:13.911627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1f328-9626-49e1-935c-cac547b4ff4e admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:13.917286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790a40c-228c-4231-a5ce-b6c3f77197d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:13.918166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790a40c-228c-4231-a5ce-b6c3f77197d6 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.918714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790a40c-228c-4231-a5ce-b6c3f77197d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.919051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790a40c-228c-4231-a5ce-b6c3f77197d6 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.919976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790a40c-228c-4231-a5ce-b6c3f77197d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:13.922291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790a40c-228c-4231-a5ce-b6c3f77197d6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:14.068132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 820/1668] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:13 2022] PUT /identity/v3/projects/286e5e2ac9014adbb153202fdb2e48aa/users/1af1b25ebf86490a979440fd0e757255/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:14.086032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5858b-9ef5-4103-87e7-fd304491577b None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:14.086323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5858b-9ef5-4103-87e7-fd304491577b None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:14.086603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5858b-9ef5-4103-87e7-fd304491577b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:14.102774 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 849/1669] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:13 2022] POST /identity/v3/users => generated 283 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:50:14.103511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790a40c-228c-4231-a5ce-b6c3f77197d6 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:14.139745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790a40c-228c-4231-a5ce-b6c3f77197d6 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:14.190182 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-3ee5858b-9ef5-4103-87e7-fd304491577b None None] MFA Rules not processed for user `1af1b25ebf86490a979440fd0e757255`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:14.191624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3ee5858b-9ef5-4103-87e7-fd304491577b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:14.283587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3cd69-58f0-4464-ae18-9ac82959df7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:14.292697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3cd69-58f0-4464-ae18-9ac82959df7f admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:14.294427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3cd69-58f0-4464-ae18-9ac82959df7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:14.294696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3cd69-58f0-4464-ae18-9ac82959df7f admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:14.296533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3cd69-58f0-4464-ae18-9ac82959df7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:14.300393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3cd69-58f0-4464-ae18-9ac82959df7f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:14.350482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 850/1670] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:14.351693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3cd69-58f0-4464-ae18-9ac82959df7f admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:14.363174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 821/1671] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:14.381054 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5858b-9ef5-4103-87e7-fd304491577b None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:14.390116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3cd69-58f0-4464-ae18-9ac82959df7f admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:14.397129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5858b-9ef5-4103-87e7-fd304491577b None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:14.409682 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4523d5-d955-431f-91bd-8db7347a23fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:14.410419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4523d5-d955-431f-91bd-8db7347a23fd admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:14.411289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4523d5-d955-431f-91bd-8db7347a23fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:14.411581 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4523d5-d955-431f-91bd-8db7347a23fd admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:14.412313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4523d5-d955-431f-91bd-8db7347a23fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:14.420143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3ce29-8e61-40eb-b0a2-7d130d9cc0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:14.420969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3ce29-8e61-40eb-b0a2-7d130d9cc0ad admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:14.421079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4523d5-d955-431f-91bd-8db7347a23fd admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:14.421270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3ce29-8e61-40eb-b0a2-7d130d9cc0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:14.421564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3ce29-8e61-40eb-b0a2-7d130d9cc0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:14.428129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3ce29-8e61-40eb-b0a2-7d130d9cc0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:14.429973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3ce29-8e61-40eb-b0a2-7d130d9cc0ad admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:14.477405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 851/1672] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:14.486353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3ce29-8e61-40eb-b0a2-7d130d9cc0ad admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:14.509162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3ce29-8e61-40eb-b0a2-7d130d9cc0ad admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:14.526221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4ec70-c439-426a-a071-83722b8f49d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:14.527661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4ec70-c439-426a-a071-83722b8f49d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:14.528317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4ec70-c439-426a-a071-83722b8f49d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:14.529711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4ec70-c439-426a-a071-83722b8f49d3 admin admin] PATH_INFO: `/v3/projects/ab5ddb315184420bb84652fb9ff409d1/users/f25737bf59a54231ba0548392c5a2bed/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:14.565013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4ec70-c439-426a-a071-83722b8f49d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab5ddb315184420bb84652fb9ff409d1, user_id=f25737bf59a54231ba0548392c5a2bed, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:14.571010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4ec70-c439-426a-a071-83722b8f49d3 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:14.597189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 822/1673] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:14 2022] POST /identity/v3/projects => generated 369 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:14.598026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4523d5-d955-431f-91bd-8db7347a23fd admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:14.696234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4523d5-d955-431f-91bd-8db7347a23fd admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:14.702815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e1dd9-86f2-4b5f-bb0a-707a459a9502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:14.703535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e1dd9-86f2-4b5f-bb0a-707a459a9502 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:14.703929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e1dd9-86f2-4b5f-bb0a-707a459a9502 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:14.704294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e1dd9-86f2-4b5f-bb0a-707a459a9502 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:14.705231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e1dd9-86f2-4b5f-bb0a-707a459a9502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:14.707321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e1dd9-86f2-4b5f-bb0a-707a459a9502 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:14.746591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 852/1674] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:14 2022] PUT /identity/v3/projects/ab5ddb315184420bb84652fb9ff409d1/users/f25737bf59a54231ba0548392c5a2bed/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:14.762026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfb95e-737c-42e7-b1b7-63afc55bc84b None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:14.762321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfb95e-737c-42e7-b1b7-63afc55bc84b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:14.762602 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfb95e-737c-42e7-b1b7-63afc55bc84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:14.828660 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-80cfb95e-737c-42e7-b1b7-63afc55bc84b None None] MFA Rules not processed for user `f25737bf59a54231ba0548392c5a2bed`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:14.829977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-80cfb95e-737c-42e7-b1b7-63afc55bc84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:14.852249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 823/1675] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:14 2022] POST /identity/v3/users => generated 291 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:14.856479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e1dd9-86f2-4b5f-bb0a-707a459a9502 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:14.894756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e1dd9-86f2-4b5f-bb0a-707a459a9502 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:14.996610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418368a-fa47-4321-af4f-affc0681abd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:14.997510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418368a-fa47-4321-af4f-affc0681abd7 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:14.997880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418368a-fa47-4321-af4f-affc0681abd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:14.998230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418368a-fa47-4321-af4f-affc0681abd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:14.999031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418368a-fa47-4321-af4f-affc0681abd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:15.001144 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418368a-fa47-4321-af4f-affc0681abd7 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:15.013680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 824/1676] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:15.017891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418368a-fa47-4321-af4f-affc0681abd7 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:15.020150 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 853/1677] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:15.029989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418368a-fa47-4321-af4f-affc0681abd7 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:15.033668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfb95e-737c-42e7-b1b7-63afc55bc84b None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:15.039728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425cea5-cdf5-48a9-bb0d-b9f6aae9f90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:15.040709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425cea5-cdf5-48a9-bb0d-b9f6aae9f90c admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:15.042939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425cea5-cdf5-48a9-bb0d-b9f6aae9f90c admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:15.043390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425cea5-cdf5-48a9-bb0d-b9f6aae9f90c admin admin] PATH_INFO: `/v3/projects/e12e3e15719b46b7bd53c0bb17349c56/users/5dc38c287e3b40fdaba2d969da8a9100/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:15.054973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfb95e-737c-42e7-b1b7-63afc55bc84b None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:15.064491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb18f2e-0e78-44bc-8841-a46d56b19a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:15.065134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb18f2e-0e78-44bc-8841-a46d56b19a61 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:15.065415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb18f2e-0e78-44bc-8841-a46d56b19a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:15.065684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb18f2e-0e78-44bc-8841-a46d56b19a61 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:15.066380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb18f2e-0e78-44bc-8841-a46d56b19a61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:15.068407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb18f2e-0e78-44bc-8841-a46d56b19a61 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:15.081319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425cea5-cdf5-48a9-bb0d-b9f6aae9f90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12e3e15719b46b7bd53c0bb17349c56, user_id=5dc38c287e3b40fdaba2d969da8a9100, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:15.087350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425cea5-cdf5-48a9-bb0d-b9f6aae9f90c admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:15.330424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 825/1678] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:15 2022] PUT /identity/v3/projects/e12e3e15719b46b7bd53c0bb17349c56/users/5dc38c287e3b40fdaba2d969da8a9100/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:15.341617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425cea5-cdf5-48a9-bb0d-b9f6aae9f90c admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:15.434861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 854/1679] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:15 2022] POST /identity/v3/projects => generated 365 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:50:15.440246 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb18f2e-0e78-44bc-8841-a46d56b19a61 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:15.510600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425cea5-cdf5-48a9-bb0d-b9f6aae9f90c admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:15.653422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb18f2e-0e78-44bc-8841-a46d56b19a61 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:15.671451 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3445012-eb7f-4157-a7b6-1a30a8d47428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:15.671664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f176e-e2b9-4447-8bae-c2a7b0cc3efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:15.672241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3445012-eb7f-4157-a7b6-1a30a8d47428 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:15.672703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3445012-eb7f-4157-a7b6-1a30a8d47428 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:15.672983 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3445012-eb7f-4157-a7b6-1a30a8d47428 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:15.673636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3445012-eb7f-4157-a7b6-1a30a8d47428 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:15.675134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f176e-e2b9-4447-8bae-c2a7b0cc3efb admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:15.675432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f176e-e2b9-4447-8bae-c2a7b0cc3efb admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:15.675604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3445012-eb7f-4157-a7b6-1a30a8d47428 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:15.675700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f176e-e2b9-4447-8bae-c2a7b0cc3efb admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:15.676411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f176e-e2b9-4447-8bae-c2a7b0cc3efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:15.678032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f176e-e2b9-4447-8bae-c2a7b0cc3efb admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:15.693179 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 826/1680] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:15.695291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f176e-e2b9-4447-8bae-c2a7b0cc3efb admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:15.718687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f176e-e2b9-4447-8bae-c2a7b0cc3efb admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:15.725862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f34ca-309c-4a3e-8c98-a06557df6c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:15.726741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f34ca-309c-4a3e-8c98-a06557df6c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:15.727263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f34ca-309c-4a3e-8c98-a06557df6c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:15.727669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f34ca-309c-4a3e-8c98-a06557df6c40 admin admin] PATH_INFO: `/v3/projects/e12e3e15719b46b7bd53c0bb17349c56/users/5dc38c287e3b40fdaba2d969da8a9100/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:15.737790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f34ca-309c-4a3e-8c98-a06557df6c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12e3e15719b46b7bd53c0bb17349c56, user_id=5dc38c287e3b40fdaba2d969da8a9100, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:15.737790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f34ca-309c-4a3e-8c98-a06557df6c40 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:15.760343 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 827/1681] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:15 2022] PUT /identity/v3/projects/e12e3e15719b46b7bd53c0bb17349c56/users/5dc38c287e3b40fdaba2d969da8a9100/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:15.764351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a04bf-cfa3-4e3e-b70b-767ba5f1dca6 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:15.765480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a04bf-cfa3-4e3e-b70b-767ba5f1dca6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:15.767975 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a04bf-cfa3-4e3e-b70b-767ba5f1dca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:15.803500 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-eb2a04bf-cfa3-4e3e-b70b-767ba5f1dca6 None None] MFA Rules not processed for user `5dc38c287e3b40fdaba2d969da8a9100`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:15.805729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-eb2a04bf-cfa3-4e3e-b70b-767ba5f1dca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:15.833394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 855/1682] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:15 2022] POST /identity/v3/users => generated 289 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:15.842116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3445012-eb7f-4157-a7b6-1a30a8d47428 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:15.860708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3445012-eb7f-4157-a7b6-1a30a8d47428 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:15.950242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c298f4-3a67-431b-8d37-138e544bd102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:15.951171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c298f4-3a67-431b-8d37-138e544bd102 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:15.951606 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c298f4-3a67-431b-8d37-138e544bd102 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:15.952235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c298f4-3a67-431b-8d37-138e544bd102 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:15.953125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c298f4-3a67-431b-8d37-138e544bd102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:15.955181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c298f4-3a67-431b-8d37-138e544bd102 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:15.976348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 856/1683] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:15.980635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c298f4-3a67-431b-8d37-138e544bd102 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:16.000482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c298f4-3a67-431b-8d37-138e544bd102 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:16.014432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1993b5-f51f-48fe-9a04-b26c761907a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:16.026618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1993b5-f51f-48fe-9a04-b26c761907a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:16.026618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1993b5-f51f-48fe-9a04-b26c761907a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:16.026618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1993b5-f51f-48fe-9a04-b26c761907a3 admin admin] PATH_INFO: `/v3/projects/bca6d67fc2c34af0af625dbb47af4a51/users/bd81b426290944c9add1ebc701a450cc/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:16.026854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 828/1684] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:16.035321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a04bf-cfa3-4e3e-b70b-767ba5f1dca6 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:16.038661 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-eb2a04bf-cfa3-4e3e-b70b-767ba5f1dca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:16.048724 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1993b5-f51f-48fe-9a04-b26c761907a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca6d67fc2c34af0af625dbb47af4a51, user_id=bd81b426290944c9add1ebc701a450cc, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:16.055321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1993b5-f51f-48fe-9a04-b26c761907a3 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:16.170465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 857/1685] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:15 2022] PUT /identity/v3/projects/bca6d67fc2c34af0af625dbb47af4a51/users/bd81b426290944c9add1ebc701a450cc/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:16.175709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1993b5-f51f-48fe-9a04-b26c761907a3 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:16.188242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a04bf-cfa3-4e3e-b70b-767ba5f1dca6 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:16.312419 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1993b5-f51f-48fe-9a04-b26c761907a3 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:16.318971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71ca6b-78d7-41ef-9995-07bb145946b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:16.320030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71ca6b-78d7-41ef-9995-07bb145946b5 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:16.320476 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71ca6b-78d7-41ef-9995-07bb145946b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:16.320762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71ca6b-78d7-41ef-9995-07bb145946b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:16.321495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71ca6b-78d7-41ef-9995-07bb145946b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:16.321952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648734d-f759-4314-9309-edf6ab6f17ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:16.322979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648734d-f759-4314-9309-edf6ab6f17ee service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:16.323374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71ca6b-78d7-41ef-9995-07bb145946b5 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:16.323727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648734d-f759-4314-9309-edf6ab6f17ee service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:16.324427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648734d-f759-4314-9309-edf6ab6f17ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:16.346861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 858/1686] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:16.347766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71ca6b-78d7-41ef-9995-07bb145946b5 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:16.363118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71ca6b-78d7-41ef-9995-07bb145946b5 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:16.370508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88625e42-886c-413b-89cd-6bf9544a8433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:16.371396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88625e42-886c-413b-89cd-6bf9544a8433 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:16.371687 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88625e42-886c-413b-89cd-6bf9544a8433 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:16.372032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88625e42-886c-413b-89cd-6bf9544a8433 admin admin] PATH_INFO: `/v3/projects/bca6d67fc2c34af0af625dbb47af4a51/users/bd81b426290944c9add1ebc701a450cc/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:16.375238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88625e42-886c-413b-89cd-6bf9544a8433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca6d67fc2c34af0af625dbb47af4a51, user_id=bd81b426290944c9add1ebc701a450cc, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:16.381452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88625e42-886c-413b-89cd-6bf9544a8433 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:16.408854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 859/1687] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:16 2022] PUT /identity/v3/projects/bca6d67fc2c34af0af625dbb47af4a51/users/bd81b426290944c9add1ebc701a450cc/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:16.433678 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648734d-f759-4314-9309-edf6ab6f17ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:16.435416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648734d-f759-4314-9309-edf6ab6f17ee service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:16.440698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83180e-be74-4af9-b424-90f6bc30c4bd None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:16.442566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83180e-be74-4af9-b424-90f6bc30c4bd None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:16.443080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83180e-be74-4af9-b424-90f6bc30c4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:16.462031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 829/1688] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:16.473947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e60162-5482-4003-a812-b5c7fce6d54f None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:16.474242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e60162-5482-4003-a812-b5c7fce6d54f None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:16.474529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e60162-5482-4003-a812-b5c7fce6d54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:16.516158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-8e83180e-be74-4af9-b424-90f6bc30c4bd None None] MFA Rules not processed for user `5c01a893d3ab42cc97508768791ff38e`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:16.517459 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8e83180e-be74-4af9-b424-90f6bc30c4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:16.551612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e6e60162-5482-4003-a812-b5c7fce6d54f None None] MFA Rules not processed for user `bd81b426290944c9add1ebc701a450cc`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:16.553039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e6e60162-5482-4003-a812-b5c7fce6d54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:16.675761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:16.703517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 860/1689] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:16.721790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544a81a-d946-4c4b-9f0f-ed58453e7e0d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:16.722085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544a81a-d946-4c4b-9f0f-ed58453e7e0d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:16.722374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544a81a-d946-4c4b-9f0f-ed58453e7e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:16.729599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 830/1690] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:16.761259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1544a81a-d946-4c4b-9f0f-ed58453e7e0d None None] MFA Rules not processed for user `5c01a893d3ab42cc97508768791ff38e`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:16.762561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1544a81a-d946-4c4b-9f0f-ed58453e7e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:16.790459 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 861/1691] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:16.813788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e60162-5482-4003-a812-b5c7fce6d54f None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:16.816358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e6e60162-5482-4003-a812-b5c7fce6d54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:16.840620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1f775-a182-4dd3-a95b-f6faf27380b6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:16.840912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1f775-a182-4dd3-a95b-f6faf27380b6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:16.841245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1f775-a182-4dd3-a95b-f6faf27380b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:16.873289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0eb1f775-a182-4dd3-a95b-f6faf27380b6 None None] MFA Rules not processed for user `6861433b9f4441fe842b506def255c58`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:16.880272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0eb1f775-a182-4dd3-a95b-f6faf27380b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:16.971649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e60162-5482-4003-a812-b5c7fce6d54f None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:17.007242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e3be1-0df5-414a-821f-1efab4a926ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:17.015827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3be1-0df5-414a-821f-1efab4a926ca service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:17.015827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3be1-0df5-414a-821f-1efab4a926ca service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:17.015827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3be1-0df5-414a-821f-1efab4a926ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:17.026140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 862/1692] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:17.026729 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1f775-a182-4dd3-a95b-f6faf27380b6 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:17.035802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e3be1-0df5-414a-821f-1efab4a926ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:17.041954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e3be1-0df5-414a-821f-1efab4a926ca service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:17.060194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 831/1693] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:17.063322 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476621a5-2fd3-4739-b2e4-712cc1a639bb None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:17.064072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476621a5-2fd3-4739-b2e4-712cc1a639bb None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:17.069799 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476621a5-2fd3-4739-b2e4-712cc1a639bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:17.111168 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1f775-a182-4dd3-a95b-f6faf27380b6 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:17.120992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60266da9-3270-48f8-9ffd-4fc74d80dd6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:17.121897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60266da9-3270-48f8-9ffd-4fc74d80dd6a service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:17.122184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60266da9-3270-48f8-9ffd-4fc74d80dd6a service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:17.122491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60266da9-3270-48f8-9ffd-4fc74d80dd6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:17.123459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-476621a5-2fd3-4739-b2e4-712cc1a639bb None None] MFA Rules not processed for user `72dc0c14d61e4bae9aaa31bf46cb1e08`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:17.124798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-476621a5-2fd3-4739-b2e4-712cc1a639bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:17.135991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60266da9-3270-48f8-9ffd-4fc74d80dd6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:17.137638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60266da9-3270-48f8-9ffd-4fc74d80dd6a service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:17.156347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 863/1694] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:17.247008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 832/1695] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:17.259208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8f865-1e85-46cf-9f6e-493986dfda7f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:17.259501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8f865-1e85-46cf-9f6e-493986dfda7f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:17.259785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8f865-1e85-46cf-9f6e-493986dfda7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:17.293175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-97b8f865-1e85-46cf-9f6e-493986dfda7f None None] MFA Rules not processed for user `72dc0c14d61e4bae9aaa31bf46cb1e08`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:17.296626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-97b8f865-1e85-46cf-9f6e-493986dfda7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:17.317532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 864/1696] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:17.344643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:17.936584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87505e2c-eefb-41f7-bb32-716641274cc1 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:17.937552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87505e2c-eefb-41f7-bb32-716641274cc1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:17.937989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87505e2c-eefb-41f7-bb32-716641274cc1 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:17.939615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 833/1697] 10.222.0.98 () {54 vars in 912 bytes} [Fri Aug 26 15:50:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:17.948993 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376564f0-18bf-466b-947d-4b48e3fb61c5 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:17.949914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376564f0-18bf-466b-947d-4b48e3fb61c5 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:17.950374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376564f0-18bf-466b-947d-4b48e3fb61c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:18.005543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-376564f0-18bf-466b-947d-4b48e3fb61c5 None None] MFA Rules not processed for user `8c6cad18359143868276eb9d79a3fd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:18.015955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-376564f0-18bf-466b-947d-4b48e3fb61c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:18.054495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341076c-7f8d-4bfb-b957-c7af70d2d06c None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:18.054786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341076c-7f8d-4bfb-b957-c7af70d2d06c None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:18.056492 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341076c-7f8d-4bfb-b957-c7af70d2d06c None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:18.057196 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 834/1698] 10.222.0.98 () {54 vars in 912 bytes} [Fri Aug 26 15:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:18.075810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7daf-747b-47f4-8e50-b1c0d96194ef None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:18.076498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7daf-747b-47f4-8e50-b1c0d96194ef None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:18.077007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7daf-747b-47f4-8e50-b1c0d96194ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:18.125557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b03b7daf-747b-47f4-8e50-b1c0d96194ef None None] MFA Rules not processed for user `8c6cad18359143868276eb9d79a3fd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:18.126862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b03b7daf-747b-47f4-8e50-b1c0d96194ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:18.160120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 865/1699] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:50:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:18.175990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376564f0-18bf-466b-947d-4b48e3fb61c5 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:18.177665 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-376564f0-18bf-466b-947d-4b48e3fb61c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:18.191943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376564f0-18bf-466b-947d-4b48e3fb61c5 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:18.196401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80852db7-a231-4269-8c76-e516c2b7dd5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:18.197226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80852db7-a231-4269-8c76-e516c2b7dd5a service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:18.197690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80852db7-a231-4269-8c76-e516c2b7dd5a service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:18.198108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80852db7-a231-4269-8c76-e516c2b7dd5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:18.210212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80852db7-a231-4269-8c76-e516c2b7dd5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:18.210212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80852db7-a231-4269-8c76-e516c2b7dd5a service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:18.217818 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 835/1700] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:50:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:18.218521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 866/1701] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:18.229018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b7daf-747b-47f4-8e50-b1c0d96194ef None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:18.230604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b03b7daf-747b-47f4-8e50-b1c0d96194ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:18.250124 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b7daf-747b-47f4-8e50-b1c0d96194ef None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:18.261314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd203d82-c3ac-40de-85f9-2466e9002a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:18.262583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203d82-c3ac-40de-85f9-2466e9002a4b service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:18.263477 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203d82-c3ac-40de-85f9-2466e9002a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:18.263999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203d82-c3ac-40de-85f9-2466e9002a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:18.272741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd203d82-c3ac-40de-85f9-2466e9002a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:18.274665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd203d82-c3ac-40de-85f9-2466e9002a4b service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:18.287848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 836/1702] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:18.534802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:18.848509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80852db7-a231-4269-8c76-e516c2b7dd5a service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:18.906797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd203d82-c3ac-40de-85f9-2466e9002a4b service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:18.927918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd203d82-c3ac-40de-85f9-2466e9002a4b service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:18.941382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85cfc8-cdd4-4bc1-9431-00af0535eb63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:18.943142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cfc8-cdd4-4bc1-9431-00af0535eb63 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:18.943774 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cfc8-cdd4-4bc1-9431-00af0535eb63 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:18.944276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cfc8-cdd4-4bc1-9431-00af0535eb63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:18.968644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80852db7-a231-4269-8c76-e516c2b7dd5a service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:18.974262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:18.976305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:18.976772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:18.977337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] PATH_INFO: `/v3/users/1af1b25ebf86490a979440fd0e757255` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:18.978422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85cfc8-cdd4-4bc1-9431-00af0535eb63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:18.979142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=1af1b25ebf86490a979440fd0e757255)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:18.980217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85cfc8-cdd4-4bc1-9431-00af0535eb63 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:18.983031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:18.992050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 837/1703] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:50:18.993318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85cfc8-cdd4-4bc1-9431-00af0535eb63 service nova] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:19.008608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85cfc8-cdd4-4bc1-9431-00af0535eb63 service nova] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:19.014883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14783e96-7d0a-4bd7-81b7-71fd72aed67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:19.015906 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14783e96-7d0a-4bd7-81b7-71fd72aed67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:19.016428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14783e96-7d0a-4bd7-81b7-71fd72aed67a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:19.017190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14783e96-7d0a-4bd7-81b7-71fd72aed67a admin admin] PATH_INFO: `/v3/users/f25737bf59a54231ba0548392c5a2bed` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:19.018733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14783e96-7d0a-4bd7-81b7-71fd72aed67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25737bf59a54231ba0548392c5a2bed)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:19.021218 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14783e96-7d0a-4bd7-81b7-71fd72aed67a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:19.311396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1af1b25ebf86490a979440fd0e757255'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:19.329666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1af1b25ebf86490a979440fd0e757255'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:19.330500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1af1b25ebf86490a979440fd0e757255'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:19.331770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1af1b25ebf86490a979440fd0e757255'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:19.436882 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1af1b25ebf86490a979440fd0e757255'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:19.455583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 867/1704] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:18 2022] DELETE /identity/v3/users/1af1b25ebf86490a979440fd0e757255 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:19.470274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-14783e96-7d0a-4bd7-81b7-71fd72aed67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25737bf59a54231ba0548392c5a2bed'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:19.474442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-14783e96-7d0a-4bd7-81b7-71fd72aed67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25737bf59a54231ba0548392c5a2bed'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:19.476785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:19.481621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:19.505333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-14783e96-7d0a-4bd7-81b7-71fd72aed67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25737bf59a54231ba0548392c5a2bed'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:19.591126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75721f-af92-43b7-8ef2-cfb999fd6dec admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:19.598739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c0844-8272-4f33-93b1-e96ebdac7e95 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:19.599297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c0844-8272-4f33-93b1-e96ebdac7e95 service placement] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:19.599913 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c0844-8272-4f33-93b1-e96ebdac7e95 service placement] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:19.600330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c0844-8272-4f33-93b1-e96ebdac7e95 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:19.602399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ad8c0844-8272-4f33-93b1-e96ebdac7e95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:19.615773 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c0844-8272-4f33-93b1-e96ebdac7e95 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:19.617522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c0844-8272-4f33-93b1-e96ebdac7e95 service placement] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:19.620455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-14783e96-7d0a-4bd7-81b7-71fd72aed67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25737bf59a54231ba0548392c5a2bed'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:19.629007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-14783e96-7d0a-4bd7-81b7-71fd72aed67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25737bf59a54231ba0548392c5a2bed'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:19.638505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 868/1705] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:19.640072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304856b1-d7ed-4c5a-8c85-fb3aa5f48f31 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:19.640579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304856b1-d7ed-4c5a-8c85-fb3aa5f48f31 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:19.641021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 838/1706] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:18 2022] DELETE /identity/v3/users/f25737bf59a54231ba0548392c5a2bed => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:19.641278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304856b1-d7ed-4c5a-8c85-fb3aa5f48f31 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:19.642645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24a2c7-0385-49be-bb26-8623d5c69dc9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:19.642942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24a2c7-0385-49be-bb26-8623d5c69dc9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:19.643296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 869/1707] 10.222.0.98 () {54 vars in 961 bytes} [Fri Aug 26 15:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:19.645431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9498941-bb79-4cc9-8357-2cddf6639e79 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:19.645925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9498941-bb79-4cc9-8357-2cddf6639e79 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:19.646447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9498941-bb79-4cc9-8357-2cddf6639e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:19.649188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24a2c7-0385-49be-bb26-8623d5c69dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:19.670792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f9498941-bb79-4cc9-8357-2cddf6639e79 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:19.672989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f9498941-bb79-4cc9-8357-2cddf6639e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:19.693337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-de24a2c7-0385-49be-bb26-8623d5c69dc9 None None] MFA Rules not processed for user `5c01a893d3ab42cc97508768791ff38e`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:19.695376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-de24a2c7-0385-49be-bb26-8623d5c69dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:19.780274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 870/1708] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:19.780274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f20463-cf2e-4c26-b728-4730d1f922f8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:19.780274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f20463-cf2e-4c26-b728-4730d1f922f8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:19.780274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f20463-cf2e-4c26-b728-4730d1f922f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:19.784702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 839/1709] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:19.789571 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107fbc4-c862-4e4d-b8f2-328d54fddf94 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:19.789896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107fbc4-c862-4e4d-b8f2-328d54fddf94 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:19.790145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107fbc4-c862-4e4d-b8f2-328d54fddf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:19.809904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-d1f20463-cf2e-4c26-b728-4730d1f922f8 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:19.812136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-d1f20463-cf2e-4c26-b728-4730d1f922f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:19.814820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-5107fbc4-c862-4e4d-b8f2-328d54fddf94 None None] MFA Rules not processed for user `b28f987c90df4c19a12bb1819dc65cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:19.817543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5107fbc4-c862-4e4d-b8f2-328d54fddf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:19.822929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 871/1710] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:19.823966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f20463-cf2e-4c26-b728-4730d1f922f8 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:19.837484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-d1f20463-cf2e-4c26-b728-4730d1f922f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:19.971358 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 840/1711] 10.222.0.98 () {58 vars in 1058 bytes} [Fri Aug 26 15:50:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:19.972275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107fbc4-c862-4e4d-b8f2-328d54fddf94 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:19.977106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f20463-cf2e-4c26-b728-4730d1f922f8 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:19.983189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107fbc4-c862-4e4d-b8f2-328d54fddf94 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:19.987757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a0b30-836a-4a4a-a47a-7d58603d608c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:19.988762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a0b30-836a-4a4a-a47a-7d58603d608c service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:19.989228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a0b30-836a-4a4a-a47a-7d58603d608c service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:19.989713 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a0b30-836a-4a4a-a47a-7d58603d608c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:19.993619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed637db6-0918-4bb5-a4df-2fec2fa62f6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:19.994265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637db6-0918-4bb5-a4df-2fec2fa62f6b service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:19.994825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637db6-0918-4bb5-a4df-2fec2fa62f6b service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:19.995252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637db6-0918-4bb5-a4df-2fec2fa62f6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:20.011818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed637db6-0918-4bb5-a4df-2fec2fa62f6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:20.014381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed637db6-0918-4bb5-a4df-2fec2fa62f6b service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:20.025077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a0b30-836a-4a4a-a47a-7d58603d608c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:20.028210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a0b30-836a-4a4a-a47a-7d58603d608c service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:20.029290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 872/1712] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:20.030107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed637db6-0918-4bb5-a4df-2fec2fa62f6b service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:20.042836 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 841/1713] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:20.060234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed637db6-0918-4bb5-a4df-2fec2fa62f6b service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:20.073616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febf6de-5acd-456c-a87d-d4d3bf3fde46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:20.074243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febf6de-5acd-456c-a87d-d4d3bf3fde46 service placement] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:20.074532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febf6de-5acd-456c-a87d-d4d3bf3fde46 service placement] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:20.074816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febf6de-5acd-456c-a87d-d4d3bf3fde46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:20.079615 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3febf6de-5acd-456c-a87d-d4d3bf3fde46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:20.107259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febf6de-5acd-456c-a87d-d4d3bf3fde46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:20.109159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febf6de-5acd-456c-a87d-d4d3bf3fde46 service placement] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:20.136586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 873/1714] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:20.564265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a0b30-836a-4a4a-a47a-7d58603d608c service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:20.565734 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ef6a0b30-836a-4a4a-a47a-7d58603d608c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:20.592989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a0b30-836a-4a4a-a47a-7d58603d608c service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:20.611232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:20.611929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:20.612488 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:20.613439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] PATH_INFO: `/v3/users/0072d0b0c39a4c2384f5c6e08c1303b2` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:20.615031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0072d0b0c39a4c2384f5c6e08c1303b2)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:20.617261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:20.791991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:20.946000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0072d0b0c39a4c2384f5c6e08c1303b2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:20.948856 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0072d0b0c39a4c2384f5c6e08c1303b2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:20.964070 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0072d0b0c39a4c2384f5c6e08c1303b2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:21.043508 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0072d0b0c39a4c2384f5c6e08c1303b2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:21.051634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0072d0b0c39a4c2384f5c6e08c1303b2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:21.069862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 842/1715] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:20 2022] DELETE /identity/v3/users/0072d0b0c39a4c2384f5c6e08c1303b2 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:21.075462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818e11b-4e1d-4309-9f23-6c133eb00998 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:21.076604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818e11b-4e1d-4309-9f23-6c133eb00998 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:21.076604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818e11b-4e1d-4309-9f23-6c133eb00998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:21.129011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-5818e11b-4e1d-4309-9f23-6c133eb00998 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:21.133640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5818e11b-4e1d-4309-9f23-6c133eb00998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:21.234079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 874/1716] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:21.246920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:21.250315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:21.356355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad81963-60fa-4a16-83ee-6d15426b827b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:21.367291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b692537-e7e3-49ac-8959-c0966f308017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:21.367291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b692537-e7e3-49ac-8959-c0966f308017 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:21.367291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b692537-e7e3-49ac-8959-c0966f308017 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:21.367291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b692537-e7e3-49ac-8959-c0966f308017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:21.372606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b692537-e7e3-49ac-8959-c0966f308017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:21.375058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b692537-e7e3-49ac-8959-c0966f308017 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:21.385845 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 843/1717] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:21.435674 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818e11b-4e1d-4309-9f23-6c133eb00998 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:21.437215 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5818e11b-4e1d-4309-9f23-6c133eb00998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:21.463835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818e11b-4e1d-4309-9f23-6c133eb00998 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:21.486536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:21.487490 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:21.491704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:21.492261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] PATH_INFO: `/v3/projects/286e5e2ac9014adbb153202fdb2e48aa` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:21.500648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=286e5e2ac9014adbb153202fdb2e48aa)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:21.502848 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:21.540542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b692537-e7e3-49ac-8959-c0966f308017 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:21.542627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3b692537-e7e3-49ac-8959-c0966f308017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:21.592409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b692537-e7e3-49ac-8959-c0966f308017 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:21.615439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:21.621406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:21.623456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:21.625130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] PATH_INFO: `/v3/projects/ab5ddb315184420bb84652fb9ff409d1` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:21.632202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab5ddb315184420bb84652fb9ff409d1)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:21.645963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:21.938169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '286e5e2ac9014adbb153202fdb2e48aa', 'request_id': 'req-30fac5c5-9120-408a-a292-21aeaa2d1850'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.080921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '286e5e2ac9014adbb153202fdb2e48aa', 'request_id': 'req-30fac5c5-9120-408a-a292-21aeaa2d1850'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.098397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '286e5e2ac9014adbb153202fdb2e48aa', 'request_id': 'req-30fac5c5-9120-408a-a292-21aeaa2d1850'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.104006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab5ddb315184420bb84652fb9ff409d1', 'request_id': 'req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.119144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] The token cache is being invalidate because project 286e5e2ac9014adbb153202fdb2e48aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:22.119351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.121463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.125194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 875/1718] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:21 2022] DELETE /identity/v3/projects/286e5e2ac9014adbb153202fdb2e48aa => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:22.126552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab5ddb315184420bb84652fb9ff409d1', 'request_id': 'req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.129121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:22.140780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:22.142565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab5ddb315184420bb84652fb9ff409d1', 'request_id': 'req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.213084 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fac5c5-9120-408a-a292-21aeaa2d1850 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:22.274236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:22.274889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:22.275197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:22.275495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] PATH_INFO: `/v3/users/5dc38c287e3b40fdaba2d969da8a9100` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:22.276711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc38c287e3b40fdaba2d969da8a9100)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:22.278648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:22.316525 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] The token cache is being invalidate because project ab5ddb315184420bb84652fb9ff409d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:22.317680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.319239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.322853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 844/1719] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:21 2022] DELETE /identity/v3/projects/ab5ddb315184420bb84652fb9ff409d1 => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:50:22.330564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:22.340567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:22.385175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1b70e-07e6-4c70-90f6-0ed890464bf7 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:22.391523 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:22.392503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:22.393063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:22.393561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] PATH_INFO: `/v3/projects/6110b6959363437c91a112be1b52b230` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:22.397727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6110b6959363437c91a112be1b52b230)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:22.400471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:22.744182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc38c287e3b40fdaba2d969da8a9100'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.764920 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc38c287e3b40fdaba2d969da8a9100'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.764920 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc38c287e3b40fdaba2d969da8a9100'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.765758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc38c287e3b40fdaba2d969da8a9100'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.783997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6110b6959363437c91a112be1b52b230', 'request_id': 'req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.793053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6110b6959363437c91a112be1b52b230', 'request_id': 'req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.802547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6110b6959363437c91a112be1b52b230', 'request_id': 'req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.963460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc38c287e3b40fdaba2d969da8a9100'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:22.979556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 876/1720] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:22 2022] DELETE /identity/v3/users/5dc38c287e3b40fdaba2d969da8a9100 => generated 0 bytes in 851 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:50:22.980767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:22.982947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:23.013536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:23.069213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] The token cache is being invalidate because project 6110b6959363437c91a112be1b52b230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:23.075420 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.075420 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.075420 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 845/1721] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:22 2022] DELETE /identity/v3/projects/6110b6959363437c91a112be1b52b230 => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:23.075420 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:23.080338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:23.097519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b38f-0ec8-4729-8d2f-226248814067 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:23.106207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656f329-019a-458c-be0a-7b625080a89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:23.107988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656f329-019a-458c-be0a-7b625080a89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:23.108451 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656f329-019a-458c-be0a-7b625080a89a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:23.109034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656f329-019a-458c-be0a-7b625080a89a admin admin] PATH_INFO: `/v3/users/bd81b426290944c9add1ebc701a450cc` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:23.111357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656f329-019a-458c-be0a-7b625080a89a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd81b426290944c9add1ebc701a450cc)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:23.113782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656f329-019a-458c-be0a-7b625080a89a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:23.160983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc75af2-01b0-4ba2-8a0a-a4ac80c9b216 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:23.166605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddbafc-121f-4f51-a879-259533ab6606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:23.167269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddbafc-121f-4f51-a879-259533ab6606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:23.167557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddbafc-121f-4f51-a879-259533ab6606 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:23.167839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddbafc-121f-4f51-a879-259533ab6606 admin admin] PATH_INFO: `/v3/users/020bd75ebcf24fe9adbc1072603020bc` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:23.169044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddbafc-121f-4f51-a879-259533ab6606 admin admin] RBAC: Authorizing `identity:delete_user(user_id=020bd75ebcf24fe9adbc1072603020bc)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:23.171583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddbafc-121f-4f51-a879-259533ab6606 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:23.409917 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-3656f329-019a-458c-be0a-7b625080a89a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd81b426290944c9add1ebc701a450cc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.439659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-3656f329-019a-458c-be0a-7b625080a89a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd81b426290944c9add1ebc701a450cc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.441410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-3656f329-019a-458c-be0a-7b625080a89a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd81b426290944c9add1ebc701a450cc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.442140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-3656f329-019a-458c-be0a-7b625080a89a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd81b426290944c9add1ebc701a450cc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.512785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-17ddbafc-121f-4f51-a879-259533ab6606 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '020bd75ebcf24fe9adbc1072603020bc'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.514262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-17ddbafc-121f-4f51-a879-259533ab6606 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '020bd75ebcf24fe9adbc1072603020bc'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.537181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-3656f329-019a-458c-be0a-7b625080a89a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd81b426290944c9add1ebc701a450cc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.554719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 877/1722] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:22 2022] DELETE /identity/v3/users/bd81b426290944c9add1ebc701a450cc => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:23.556503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c4952-0d07-405a-bbb8-5426bd5307a8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:23.557049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c4952-0d07-405a-bbb8-5426bd5307a8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:23.570253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c4952-0d07-405a-bbb8-5426bd5307a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:23.580530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-17ddbafc-121f-4f51-a879-259533ab6606 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '020bd75ebcf24fe9adbc1072603020bc'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.627569 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-074c4952-0d07-405a-bbb8-5426bd5307a8 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:23.629502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-074c4952-0d07-405a-bbb8-5426bd5307a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:23.708510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-17ddbafc-121f-4f51-a879-259533ab6606 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '020bd75ebcf24fe9adbc1072603020bc'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.718028 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-17ddbafc-121f-4f51-a879-259533ab6606 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '020bd75ebcf24fe9adbc1072603020bc'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:23.726022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 846/1723] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:23 2022] DELETE /identity/v3/users/020bd75ebcf24fe9adbc1072603020bc => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:23.730651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b337c-d7a1-4e61-b987-f4db67160fb6 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:23.731422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b337c-d7a1-4e61-b987-f4db67160fb6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:23.732498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b337c-d7a1-4e61-b987-f4db67160fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:23.792263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-365b337c-d7a1-4e61-b987-f4db67160fb6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:23.792263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-365b337c-d7a1-4e61-b987-f4db67160fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:23.898409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 878/1724] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:23.899648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c4952-0d07-405a-bbb8-5426bd5307a8 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:23.901939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 847/1725] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:23.903555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-074c4952-0d07-405a-bbb8-5426bd5307a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:23.908254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74182c1c-171c-49b7-8362-18f0d260166d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:23.913073 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74182c1c-171c-49b7-8362-18f0d260166d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:23.913073 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74182c1c-171c-49b7-8362-18f0d260166d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:23.953543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c4952-0d07-405a-bbb8-5426bd5307a8 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:23.975675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:23.978131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:23.978131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:23.978131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] PATH_INFO: `/v3/projects/e12e3e15719b46b7bd53c0bb17349c56` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:23.983709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e12e3e15719b46b7bd53c0bb17349c56)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:23.989369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-74182c1c-171c-49b7-8362-18f0d260166d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:23.990148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:23.990684 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-74182c1c-171c-49b7-8362-18f0d260166d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:24.017822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 848/1726] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:24.020582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74182c1c-171c-49b7-8362-18f0d260166d None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:24.024145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-74182c1c-171c-49b7-8362-18f0d260166d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:24.124391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74182c1c-171c-49b7-8362-18f0d260166d None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:24.132651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fe3b5-d0f9-4569-bd29-5c81a00bc580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:24.132651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fe3b5-d0f9-4569-bd29-5c81a00bc580 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.132651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fe3b5-d0f9-4569-bd29-5c81a00bc580 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.132651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fe3b5-d0f9-4569-bd29-5c81a00bc580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.144619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803fe3b5-d0f9-4569-bd29-5c81a00bc580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:24.146583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803fe3b5-d0f9-4569-bd29-5c81a00bc580 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:24.171338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e12e3e15719b46b7bd53c0bb17349c56', 'request_id': 'req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:24.177177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 849/1727] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:24.178504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fe3b5-d0f9-4569-bd29-5c81a00bc580 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:24.210302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fe3b5-d0f9-4569-bd29-5c81a00bc580 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:24.269825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e12e3e15719b46b7bd53c0bb17349c56', 'request_id': 'req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:24.291161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e12e3e15719b46b7bd53c0bb17349c56', 'request_id': 'req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:24.310376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db528b-5554-49a7-8d1e-f7bd4177a5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:24.310376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db528b-5554-49a7-8d1e-f7bd4177a5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.310376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db528b-5554-49a7-8d1e-f7bd4177a5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.310376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db528b-5554-49a7-8d1e-f7bd4177a5d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.310895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db528b-5554-49a7-8d1e-f7bd4177a5d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:24.310895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db528b-5554-49a7-8d1e-f7bd4177a5d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:24.310895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db528b-5554-49a7-8d1e-f7bd4177a5d9 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:24.328790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 850/1728] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:24.330644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] The token cache is being invalidate because project e12e3e15719b46b7bd53c0bb17349c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:24.333288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:24.335475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db528b-5554-49a7-8d1e-f7bd4177a5d9 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:24.336029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1170caa8-a2e5-4bb2-b1b1-ab4cd56ae68c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:24.339584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 879/1729] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:23 2022] DELETE /identity/v3/projects/e12e3e15719b46b7bd53c0bb17349c56 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:24.345205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-19db528b-5554-49a7-8d1e-f7bd4177a5d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:24.349157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0c817-a4be-477a-9a54-b9fea6cf0a7d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.349521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0c817-a4be-477a-9a54-b9fea6cf0a7d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.349808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0c817-a4be-477a-9a54-b9fea6cf0a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.377791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db528b-5554-49a7-8d1e-f7bd4177a5d9 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:24.393071 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-5bf0c817-a4be-477a-9a54-b9fea6cf0a7d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:24.394432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5bf0c817-a4be-477a-9a54-b9fea6cf0a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:24.395200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f5981-1356-44eb-8611-d67953a3b383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:24.395967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f5981-1356-44eb-8611-d67953a3b383 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.396256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f5981-1356-44eb-8611-d67953a3b383 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.396533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f5981-1356-44eb-8611-d67953a3b383 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.397172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f5981-1356-44eb-8611-d67953a3b383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:24.403045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f5981-1356-44eb-8611-d67953a3b383 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:24.429974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 880/1730] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:24.436536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850deb8-30b3-499f-9a9c-b9f776bf50ee None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.438217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850deb8-30b3-499f-9a9c-b9f776bf50ee None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.439666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850deb8-30b3-499f-9a9c-b9f776bf50ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.484879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-6850deb8-30b3-499f-9a9c-b9f776bf50ee None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:24.486901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6850deb8-30b3-499f-9a9c-b9f776bf50ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:24.495642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 881/1731] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:24.503230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850deb8-30b3-499f-9a9c-b9f776bf50ee None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:24.525856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850deb8-30b3-499f-9a9c-b9f776bf50ee None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:24.543043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f988-9104-4557-8a84-a61df1b4c877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:24.556182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f988-9104-4557-8a84-a61df1b4c877 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.556182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f988-9104-4557-8a84-a61df1b4c877 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.556182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f988-9104-4557-8a84-a61df1b4c877 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.556182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9f988-9104-4557-8a84-a61df1b4c877 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:24.556182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9f988-9104-4557-8a84-a61df1b4c877 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:24.556182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9f988-9104-4557-8a84-a61df1b4c877 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:24.570442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 882/1732] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:50:24.571252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f988-9104-4557-8a84-a61df1b4c877 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:24.586055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f988-9104-4557-8a84-a61df1b4c877 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:24.605029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aea80f-4910-4a52-8802-ba89175d7762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:24.605029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea80f-4910-4a52-8802-ba89175d7762 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.605029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea80f-4910-4a52-8802-ba89175d7762 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.605029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea80f-4910-4a52-8802-ba89175d7762 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.605437 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aea80f-4910-4a52-8802-ba89175d7762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:24.605694 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aea80f-4910-4a52-8802-ba89175d7762 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:24.616648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 851/1733] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:24 2022] POST /identity/v3/projects => generated 373 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:24.624920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f5981-1356-44eb-8611-d67953a3b383 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:24.721655 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f5981-1356-44eb-8611-d67953a3b383 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:24.728570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0bdab-935f-44ec-913b-fb7877c2dec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:24.729272 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0bdab-935f-44ec-913b-fb7877c2dec1 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.729556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0bdab-935f-44ec-913b-fb7877c2dec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.729832 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0bdab-935f-44ec-913b-fb7877c2dec1 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.730469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0bdab-935f-44ec-913b-fb7877c2dec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:24.732693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0bdab-935f-44ec-913b-fb7877c2dec1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:24.797336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 883/1734] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:24 2022] POST /identity/v3/projects => generated 381 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:24.801816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aea80f-4910-4a52-8802-ba89175d7762 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:24.939059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 852/1735] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:24 2022] POST /identity/v3/users => generated 287 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:24.944180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0bdab-935f-44ec-913b-fb7877c2dec1 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:24.946876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3bd0bdab-935f-44ec-913b-fb7877c2dec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:24.968229 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aea80f-4910-4a52-8802-ba89175d7762 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:24.982321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ade7a-3bdc-435d-843a-dd93b0351ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:24.982957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ade7a-3bdc-435d-843a-dd93b0351ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.983248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ade7a-3bdc-435d-843a-dd93b0351ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.983532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ade7a-3bdc-435d-843a-dd93b0351ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.984317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ade7a-3bdc-435d-843a-dd93b0351ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:24.986447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ade7a-3bdc-435d-843a-dd93b0351ad5 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:24.994447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0bdab-935f-44ec-913b-fb7877c2dec1 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:25.016016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:25.020631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:25.021599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:25.023056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] PATH_INFO: `/v3/projects/bca6d67fc2c34af0af625dbb47af4a51` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:25.026929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca6d67fc2c34af0af625dbb47af4a51)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:25.029067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:25.141323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 884/1736] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:24 2022] POST /identity/v3/users => generated 291 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:25.150332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ade7a-3bdc-435d-843a-dd93b0351ad5 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:25.156524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ade7a-3bdc-435d-843a-dd93b0351ad5 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:25.175366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8644fb-e95c-492b-898f-b297aced2445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:25.176064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8644fb-e95c-492b-898f-b297aced2445 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:25.176334 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8644fb-e95c-492b-898f-b297aced2445 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:25.176592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8644fb-e95c-492b-898f-b297aced2445 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:25.177351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8644fb-e95c-492b-898f-b297aced2445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:25.179035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8644fb-e95c-492b-898f-b297aced2445 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:25.194544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 885/1737] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:25.194544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8644fb-e95c-492b-898f-b297aced2445 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:25.194829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9f8644fb-e95c-492b-898f-b297aced2445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:25.208657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8644fb-e95c-492b-898f-b297aced2445 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:25.215495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:25.216945 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:25.217767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:25.218053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] PATH_INFO: `/v3/projects/00ee3af9e5784b81a51565cd2fbd77ca` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:25.219518 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00ee3af9e5784b81a51565cd2fbd77ca)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:25.221517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:25.252408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca6d67fc2c34af0af625dbb47af4a51', 'request_id': 'req-b6322455-e44c-415c-a212-b8ef19a0c029'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:25.259167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca6d67fc2c34af0af625dbb47af4a51', 'request_id': 'req-b6322455-e44c-415c-a212-b8ef19a0c029'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:25.267631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca6d67fc2c34af0af625dbb47af4a51', 'request_id': 'req-b6322455-e44c-415c-a212-b8ef19a0c029'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:25.394501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] The token cache is being invalidate because project bca6d67fc2c34af0af625dbb47af4a51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:25.394823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:25.396591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:25.398881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 853/1738] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:24 2022] DELETE /identity/v3/projects/bca6d67fc2c34af0af625dbb47af4a51 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:25.405100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:25.409943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:25.417512 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00ee3af9e5784b81a51565cd2fbd77ca', 'request_id': 'req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:25.521778 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6322455-e44c-415c-a212-b8ef19a0c029 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:25.617598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00ee3af9e5784b81a51565cd2fbd77ca', 'request_id': 'req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:25.630297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00ee3af9e5784b81a51565cd2fbd77ca', 'request_id': 'req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:25.631145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f02c6-4d3a-4923-90a7-c9116ff17f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:25.631644 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f02c6-4d3a-4923-90a7-c9116ff17f6d admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:25.631961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f02c6-4d3a-4923-90a7-c9116ff17f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:25.634559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f02c6-4d3a-4923-90a7-c9116ff17f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:25.639701 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f02c6-4d3a-4923-90a7-c9116ff17f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:25.644890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f02c6-4d3a-4923-90a7-c9116ff17f6d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:25.666409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 854/1739] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:25.670348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f02c6-4d3a-4923-90a7-c9116ff17f6d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:25.674092 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] The token cache is being invalidate because project 00ee3af9e5784b81a51565cd2fbd77ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:25.677694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9f5f02c6-4d3a-4923-90a7-c9116ff17f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:25.677819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:25.677819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:25.679398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 886/1740] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:25 2022] DELETE /identity/v3/projects/00ee3af9e5784b81a51565cd2fbd77ca => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:25.681854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:25.689200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:25.702460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f02c6-4d3a-4923-90a7-c9116ff17f6d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:25.712858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfdcc2-a6f4-4fe9-bacc-91a12c361aa2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:25.719506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e63b9e-c90c-4a93-a443-069b4400f561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:25.720284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e63b9e-c90c-4a93-a443-069b4400f561 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:25.720576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e63b9e-c90c-4a93-a443-069b4400f561 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:25.720867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e63b9e-c90c-4a93-a443-069b4400f561 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:25.721590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e63b9e-c90c-4a93-a443-069b4400f561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:25.722427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579a11c-0a4f-4f6b-9e5f-28d3e56efab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:25.723231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e63b9e-c90c-4a93-a443-069b4400f561 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:25.723492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579a11c-0a4f-4f6b-9e5f-28d3e56efab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:25.723992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579a11c-0a4f-4f6b-9e5f-28d3e56efab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:25.724493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579a11c-0a4f-4f6b-9e5f-28d3e56efab8 admin admin] PATH_INFO: `/v3/users/af36ee65e4c54dc38d9f876e093b1c2f` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:25.726016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579a11c-0a4f-4f6b-9e5f-28d3e56efab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af36ee65e4c54dc38d9f876e093b1c2f)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:25.728252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579a11c-0a4f-4f6b-9e5f-28d3e56efab8 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:25.749770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 855/1741] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:25.751119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3a1a2-3f5c-43f9-bf58-74929d59c895 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:25.751415 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3a1a2-3f5c-43f9-bf58-74929d59c895 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:25.751702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3a1a2-3f5c-43f9-bf58-74929d59c895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:25.775469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-9cd3a1a2-3f5c-43f9-bf58-74929d59c895 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:25.777270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9cd3a1a2-3f5c-43f9-bf58-74929d59c895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:25.793535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 856/1742] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:25.793535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3a1a2-3f5c-43f9-bf58-74929d59c895 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:25.795597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9cd3a1a2-3f5c-43f9-bf58-74929d59c895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:25.816359 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3a1a2-3f5c-43f9-bf58-74929d59c895 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:25.828435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51c479-dcc9-4fb8-91fa-171f4b8b6e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:25.828435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51c479-dcc9-4fb8-91fa-171f4b8b6e7f admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:25.828435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51c479-dcc9-4fb8-91fa-171f4b8b6e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:25.828708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51c479-dcc9-4fb8-91fa-171f4b8b6e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:25.829536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51c479-dcc9-4fb8-91fa-171f4b8b6e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:25.831235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51c479-dcc9-4fb8-91fa-171f4b8b6e7f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:25.846543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 857/1743] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:25.846543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e19744-521d-4db1-8279-b74642b4249f None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:25.847137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e19744-521d-4db1-8279-b74642b4249f None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:25.848199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e19744-521d-4db1-8279-b74642b4249f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:25.875061 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-51e19744-521d-4db1-8279-b74642b4249f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:25.876462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-51e19744-521d-4db1-8279-b74642b4249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:25.896067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 858/1744] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:25.896067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e19744-521d-4db1-8279-b74642b4249f None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:25.897403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-51e19744-521d-4db1-8279-b74642b4249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:25.912760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e19744-521d-4db1-8279-b74642b4249f None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:25.925825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac6adb-7e66-4d10-9d61-95ed72b0200b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:25.927480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6adb-7e66-4d10-9d61-95ed72b0200b admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:25.927480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6adb-7e66-4d10-9d61-95ed72b0200b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:25.928244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6adb-7e66-4d10-9d61-95ed72b0200b admin admin] PATH_INFO: `/v3/projects/d0c8f1dcba8f46dfb7ce7ce9e099bbc3/users/2f28260e27934437a1d2122849a0669a/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:25.950102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac6adb-7e66-4d10-9d61-95ed72b0200b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c8f1dcba8f46dfb7ce7ce9e099bbc3, user_id=2f28260e27934437a1d2122849a0669a, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:25.953546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac6adb-7e66-4d10-9d61-95ed72b0200b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:26.151578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:26.152139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 859/1745] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:25 2022] PUT /identity/v3/projects/d0c8f1dcba8f46dfb7ce7ce9e099bbc3/users/2f28260e27934437a1d2122849a0669a/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:26.157495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd264e1e-d965-43e9-bd56-b9d960d28f7b None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:26.157908 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd264e1e-d965-43e9-bd56-b9d960d28f7b None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:26.158865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd264e1e-d965-43e9-bd56-b9d960d28f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:26.203325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a579a11c-0a4f-4f6b-9e5f-28d3e56efab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af36ee65e4c54dc38d9f876e093b1c2f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:26.257152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-fd264e1e-d965-43e9-bd56-b9d960d28f7b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:26.261059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-fd264e1e-d965-43e9-bd56-b9d960d28f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:26.265136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a579a11c-0a4f-4f6b-9e5f-28d3e56efab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af36ee65e4c54dc38d9f876e093b1c2f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:26.267631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a579a11c-0a4f-4f6b-9e5f-28d3e56efab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af36ee65e4c54dc38d9f876e093b1c2f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:26.272278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a579a11c-0a4f-4f6b-9e5f-28d3e56efab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af36ee65e4c54dc38d9f876e093b1c2f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:26.382299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 860/1746] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:26.386651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd264e1e-d965-43e9-bd56-b9d960d28f7b None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:26.389464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-fd264e1e-d965-43e9-bd56-b9d960d28f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:26.397415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a579a11c-0a4f-4f6b-9e5f-28d3e56efab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af36ee65e4c54dc38d9f876e093b1c2f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:26.412966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd264e1e-d965-43e9-bd56-b9d960d28f7b None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:26.414952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 887/1747] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:25 2022] DELETE /identity/v3/users/af36ee65e4c54dc38d9f876e093b1c2f => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:26.418026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73c2ee-d909-4ae5-8960-18716aa9402f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:26.418026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73c2ee-d909-4ae5-8960-18716aa9402f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:26.418026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73c2ee-d909-4ae5-8960-18716aa9402f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:26.445226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-bc73c2ee-d909-4ae5-8960-18716aa9402f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:26.447697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bc73c2ee-d909-4ae5-8960-18716aa9402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:26.530052 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0e2d7-d20f-4a74-ac09-a34900037a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:26.533780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0e2d7-d20f-4a74-ac09-a34900037a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:26.533780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0e2d7-d20f-4a74-ac09-a34900037a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:26.533780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0e2d7-d20f-4a74-ac09-a34900037a92 admin admin] PATH_INFO: `/v3/projects/af0fa4f49c3743f8b3c2b8f901997b8c/users/3008ae15fffc4ae1b353ce4fe2de9bfc/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:26.541197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 888/1748] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:26.547762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7b671-f995-4823-af3f-03899ef796ff None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:26.548209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7b671-f995-4823-af3f-03899ef796ff None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:26.548584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7b671-f995-4823-af3f-03899ef796ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:26.556644 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0e2d7-d20f-4a74-ac09-a34900037a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0fa4f49c3743f8b3c2b8f901997b8c, user_id=3008ae15fffc4ae1b353ce4fe2de9bfc, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:26.561263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0e2d7-d20f-4a74-ac09-a34900037a92 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:26.602225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0fc7b671-f995-4823-af3f-03899ef796ff None None] MFA Rules not processed for user `2f28260e27934437a1d2122849a0669a`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:26.603689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0fc7b671-f995-4823-af3f-03899ef796ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:26.678533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 861/1749] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:26 2022] PUT /identity/v3/projects/af0fa4f49c3743f8b3c2b8f901997b8c/users/3008ae15fffc4ae1b353ce4fe2de9bfc/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:26.679704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0e2d7-d20f-4a74-ac09-a34900037a92 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:26.738218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 889/1750] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:26.742640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9de47-acac-46a1-9c9e-803336236111 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:26.743192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9de47-acac-46a1-9c9e-803336236111 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:26.743690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9de47-acac-46a1-9c9e-803336236111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:26.778806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-7fb9de47-acac-46a1-9c9e-803336236111 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:26.781371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-7fb9de47-acac-46a1-9c9e-803336236111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:26.846463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0e2d7-d20f-4a74-ac09-a34900037a92 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:26.852282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c00c0-e7ff-4a78-9aab-f4fadad593ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:26.853014 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c00c0-e7ff-4a78-9aab-f4fadad593ec admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:26.853379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c00c0-e7ff-4a78-9aab-f4fadad593ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:26.854076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c00c0-e7ff-4a78-9aab-f4fadad593ec admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:26.856776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c00c0-e7ff-4a78-9aab-f4fadad593ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:26.857721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c00c0-e7ff-4a78-9aab-f4fadad593ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:26.860268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c00c0-e7ff-4a78-9aab-f4fadad593ec admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:26.880317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 862/1751] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:26.880317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c00c0-e7ff-4a78-9aab-f4fadad593ec admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:26.880578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 890/1752] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:26.880578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98a622-c32f-4a87-8ddf-d428fa91e58b None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:26.880578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98a622-c32f-4a87-8ddf-d428fa91e58b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:26.880578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98a622-c32f-4a87-8ddf-d428fa91e58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:26.893616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c00c0-e7ff-4a78-9aab-f4fadad593ec admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:26.898902 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9b9c9-4965-4f7a-a2ef-5584f1ee93f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:26.899530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9b9c9-4965-4f7a-a2ef-5584f1ee93f1 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:26.899824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9b9c9-4965-4f7a-a2ef-5584f1ee93f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:26.901252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9b9c9-4965-4f7a-a2ef-5584f1ee93f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:26.902002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9b9c9-4965-4f7a-a2ef-5584f1ee93f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:26.902316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9b9c9-4965-4f7a-a2ef-5584f1ee93f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:26.905111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9b9c9-4965-4f7a-a2ef-5584f1ee93f1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:26.916093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-bc98a622-c32f-4a87-8ddf-d428fa91e58b None None] MFA Rules not processed for user `3008ae15fffc4ae1b353ce4fe2de9bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:26.917398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bc98a622-c32f-4a87-8ddf-d428fa91e58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:26.931180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 863/1753] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:26.931180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9b9c9-4965-4f7a-a2ef-5584f1ee93f1 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:26.933639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3da9b9c9-4965-4f7a-a2ef-5584f1ee93f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:27.059927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:27.075747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 891/1754] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:27.077608 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9209b1-d8dd-4ade-ad5b-7b1d764cd20f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:27.078027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9209b1-d8dd-4ade-ad5b-7b1d764cd20f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:27.078440 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9209b1-d8dd-4ade-ad5b-7b1d764cd20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:27.086048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9b9c9-4965-4f7a-a2ef-5584f1ee93f1 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:27.093290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e453-194a-4089-991d-2dbc14eeb988 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:27.094153 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e453-194a-4089-991d-2dbc14eeb988 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:27.094444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e453-194a-4089-991d-2dbc14eeb988 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:27.094720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e453-194a-4089-991d-2dbc14eeb988 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:27.133582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-8c9209b1-d8dd-4ade-ad5b-7b1d764cd20f None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:27.134708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8c9209b1-d8dd-4ade-ad5b-7b1d764cd20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:27.158197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082e453-194a-4089-991d-2dbc14eeb988 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:27.160093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082e453-194a-4089-991d-2dbc14eeb988 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:27.170238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 864/1755] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:27.171897 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e453-194a-4089-991d-2dbc14eeb988 service nova] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:27.191556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e453-194a-4089-991d-2dbc14eeb988 service nova] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:27.231278 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e170308-1019-419b-aed7-55f23b8de0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:27.231800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170308-1019-419b-aed7-55f23b8de0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:27.231800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170308-1019-419b-aed7-55f23b8de0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:27.232834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170308-1019-419b-aed7-55f23b8de0d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:27.233481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e170308-1019-419b-aed7-55f23b8de0d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:27.247436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e170308-1019-419b-aed7-55f23b8de0d3 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:27.280200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 892/1756] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:50:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:27.284642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9209b1-d8dd-4ade-ad5b-7b1d764cd20f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:27.284642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8c9209b1-d8dd-4ade-ad5b-7b1d764cd20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:27.393162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 865/1757] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:27 2022] POST /identity/v3/projects => generated 375 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:27.395714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e170308-1019-419b-aed7-55f23b8de0d3 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:27.486687 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9209b1-d8dd-4ade-ad5b-7b1d764cd20f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:27.586984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c29c1-dcd7-490e-8b5f-3e7987e6cf03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:27.587769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c29c1-dcd7-490e-8b5f-3e7987e6cf03 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:27.588196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c29c1-dcd7-490e-8b5f-3e7987e6cf03 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:27.589046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c29c1-dcd7-490e-8b5f-3e7987e6cf03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:27.601790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c29c1-dcd7-490e-8b5f-3e7987e6cf03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:27.603502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c29c1-dcd7-490e-8b5f-3e7987e6cf03 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:27.605588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e170308-1019-419b-aed7-55f23b8de0d3 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:27.610711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b01e71-73fb-4e42-8b82-083b968762c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:27.611347 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01e71-73fb-4e42-8b82-083b968762c5 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:27.611633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01e71-73fb-4e42-8b82-083b968762c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:27.611941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01e71-73fb-4e42-8b82-083b968762c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:27.612584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b01e71-73fb-4e42-8b82-083b968762c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:27.614515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b01e71-73fb-4e42-8b82-083b968762c5 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:27.616611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 893/1758] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:27.618106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c29c1-dcd7-490e-8b5f-3e7987e6cf03 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:27.628678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c29c1-dcd7-490e-8b5f-3e7987e6cf03 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:27.635481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61368183-062c-48cf-87d8-f8de83255434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:27.636061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61368183-062c-48cf-87d8-f8de83255434 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:27.637261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61368183-062c-48cf-87d8-f8de83255434 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:27.637556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61368183-062c-48cf-87d8-f8de83255434 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:27.638749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61368183-062c-48cf-87d8-f8de83255434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:27.641214 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61368183-062c-48cf-87d8-f8de83255434 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:27.748539 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 866/1759] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:27 2022] POST /identity/v3/projects => generated 365 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:27.749569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b01e71-73fb-4e42-8b82-083b968762c5 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:27.810236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 894/1760] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:27 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:27.811489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61368183-062c-48cf-87d8-f8de83255434 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:27.815378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-61368183-062c-48cf-87d8-f8de83255434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:27.912396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b01e71-73fb-4e42-8b82-083b968762c5 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:27.955376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61368183-062c-48cf-87d8-f8de83255434 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:27.963935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:27.964700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:27.964962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:27.966049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] PATH_INFO: `/v3/users/2f28260e27934437a1d2122849a0669a` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:27.967041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f28260e27934437a1d2122849a0669a)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:27.969382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:28.037808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443c5c1-3c2a-4fbf-801b-82ccb731c764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:28.038836 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443c5c1-3c2a-4fbf-801b-82ccb731c764 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:28.039287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443c5c1-3c2a-4fbf-801b-82ccb731c764 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:28.040728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443c5c1-3c2a-4fbf-801b-82ccb731c764 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:28.041733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443c5c1-3c2a-4fbf-801b-82ccb731c764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:28.044284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443c5c1-3c2a-4fbf-801b-82ccb731c764 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:28.319028 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 867/1761] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:27 2022] POST /identity/v3/users => generated 288 bytes in 568 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:28.319028 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443c5c1-3c2a-4fbf-801b-82ccb731c764 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:28.332640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f28260e27934437a1d2122849a0669a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:28.348664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443c5c1-3c2a-4fbf-801b-82ccb731c764 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:28.356511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9572a8-f4af-4d45-91e1-49a5d8f235b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:28.357365 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9572a8-f4af-4d45-91e1-49a5d8f235b1 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:28.357658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9572a8-f4af-4d45-91e1-49a5d8f235b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:28.357944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9572a8-f4af-4d45-91e1-49a5d8f235b1 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:28.358662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9572a8-f4af-4d45-91e1-49a5d8f235b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:28.361000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9572a8-f4af-4d45-91e1-49a5d8f235b1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:28.392155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f28260e27934437a1d2122849a0669a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:28.393631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f28260e27934437a1d2122849a0669a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:28.395915 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f28260e27934437a1d2122849a0669a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:28.478761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 868/1762] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:28 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:28.478761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9572a8-f4af-4d45-91e1-49a5d8f235b1 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:28.482203 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4c9572a8-f4af-4d45-91e1-49a5d8f235b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:28.491915 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f28260e27934437a1d2122849a0669a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:28.500730 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9572a8-f4af-4d45-91e1-49a5d8f235b1 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:28.510695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f32a1-302a-4970-99be-40bced7cd4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:28.511435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f32a1-302a-4970-99be-40bced7cd4af admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:28.511824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f32a1-302a-4970-99be-40bced7cd4af admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:28.512132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f32a1-302a-4970-99be-40bced7cd4af admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:28.512768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f32a1-302a-4970-99be-40bced7cd4af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:28.517197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 895/1763] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:27 2022] DELETE /identity/v3/users/2f28260e27934437a1d2122849a0669a => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:28.517197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:28.520545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:28.521015 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f32a1-302a-4970-99be-40bced7cd4af admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:28.665423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 869/1764] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:28 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:28.666112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f32a1-302a-4970-99be-40bced7cd4af admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:28.669647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a79f32a1-302a-4970-99be-40bced7cd4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:28.691370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f32a1-302a-4970-99be-40bced7cd4af admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:28.746044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb74462-f228-4dea-a4cc-cb3439d01198 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:28.766185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80355a-dadb-4847-a73b-968ce49e3a27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:28.766185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80355a-dadb-4847-a73b-968ce49e3a27 service swift] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:28.766185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80355a-dadb-4847-a73b-968ce49e3a27 service swift] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:28.766185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80355a-dadb-4847-a73b-968ce49e3a27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:28.766731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2a80355a-dadb-4847-a73b-968ce49e3a27 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:28.769441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cd1ce-9d1b-45df-ada8-1f3fb2c0bda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:28.770239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cd1ce-9d1b-45df-ada8-1f3fb2c0bda4 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:28.770528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cd1ce-9d1b-45df-ada8-1f3fb2c0bda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:28.770850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cd1ce-9d1b-45df-ada8-1f3fb2c0bda4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:28.771708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cd1ce-9d1b-45df-ada8-1f3fb2c0bda4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:28.773367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cd1ce-9d1b-45df-ada8-1f3fb2c0bda4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:28.788313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 870/1765] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:28.788313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7ea27-fdc9-403b-92eb-e5b3c9230324 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:28.788313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7ea27-fdc9-403b-92eb-e5b3c9230324 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:28.788313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7ea27-fdc9-403b-92eb-e5b3c9230324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:28.813254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-d0d7ea27-fdc9-403b-92eb-e5b3c9230324 None None] MFA Rules not processed for user `8c6cad18359143868276eb9d79a3fd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:28.814644 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d0d7ea27-fdc9-403b-92eb-e5b3c9230324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:28.840999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80355a-dadb-4847-a73b-968ce49e3a27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:28.844007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80355a-dadb-4847-a73b-968ce49e3a27 service swift] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:28.853780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 896/1766] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:50:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:50:28.855077 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80355a-dadb-4847-a73b-968ce49e3a27 service swift] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:28.860099 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2a80355a-dadb-4847-a73b-968ce49e3a27 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:28.889452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80355a-dadb-4847-a73b-968ce49e3a27 service swift] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:28.889452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d151be4-55b5-461e-81d3-7c72d06d380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:28.889452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d151be4-55b5-461e-81d3-7c72d06d380c admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:28.889452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d151be4-55b5-461e-81d3-7c72d06d380c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:28.890236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d151be4-55b5-461e-81d3-7c72d06d380c admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:28.890236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d151be4-55b5-461e-81d3-7c72d06d380c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:28.890236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d151be4-55b5-461e-81d3-7c72d06d380c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:28.906948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 897/1767] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:28.907799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3d58f-292d-430e-b48b-65f777d6605d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:28.908252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3d58f-292d-430e-b48b-65f777d6605d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:28.908632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3d58f-292d-430e-b48b-65f777d6605d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:28.913796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 871/1768] 10.222.0.67 () {58 vars in 1015 bytes} [Fri Aug 26 15:50:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:28.915629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7ea27-fdc9-403b-92eb-e5b3c9230324 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:28.947968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7ea27-fdc9-403b-92eb-e5b3c9230324 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:28.955152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-4fb3d58f-292d-430e-b48b-65f777d6605d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:28.964273 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4fb3d58f-292d-430e-b48b-65f777d6605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:28.966366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3977ed-d507-4d87-9b07-9722218f77ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:28.967288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3977ed-d507-4d87-9b07-9722218f77ba admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:28.967657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3977ed-d507-4d87-9b07-9722218f77ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:28.967980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3977ed-d507-4d87-9b07-9722218f77ba admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:28.978559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 898/1769] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:28.979732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3d58f-292d-430e-b48b-65f777d6605d None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:28.992042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3977ed-d507-4d87-9b07-9722218f77ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:28.993819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3977ed-d507-4d87-9b07-9722218f77ba admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:28.994061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3d58f-292d-430e-b48b-65f777d6605d None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:29.002835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3b6ff-010c-4d51-97e5-5c0d7d7fc5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:29.003756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3b6ff-010c-4d51-97e5-5c0d7d7fc5bb admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:29.004326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3b6ff-010c-4d51-97e5-5c0d7d7fc5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:29.004831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3b6ff-010c-4d51-97e5-5c0d7d7fc5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:29.005690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3b6ff-010c-4d51-97e5-5c0d7d7fc5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:29.008354 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3b6ff-010c-4d51-97e5-5c0d7d7fc5bb admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:29.025076 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 899/1770] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:29.026323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3b6ff-010c-4d51-97e5-5c0d7d7fc5bb admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:29.029800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-86b3b6ff-010c-4d51-97e5-5c0d7d7fc5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:29.055508 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 872/1771] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:29.055508 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3977ed-d507-4d87-9b07-9722218f77ba admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:29.055809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3b6ff-010c-4d51-97e5-5c0d7d7fc5bb admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:29.062297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:29.063226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:29.063723 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:29.064218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] PATH_INFO: `/v3/projects/6198f7e071e54f0f8c5186f5b031a7b4` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:29.066149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6198f7e071e54f0f8c5186f5b031a7b4)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:29.070548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:29.101495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3977ed-d507-4d87-9b07-9722218f77ba admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:29.135300 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3708bb2-d565-4caf-a93a-1db4cd53c00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:29.144304 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3708bb2-d565-4caf-a93a-1db4cd53c00d admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:29.148729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3708bb2-d565-4caf-a93a-1db4cd53c00d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:29.148729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3708bb2-d565-4caf-a93a-1db4cd53c00d admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:29.148729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3708bb2-d565-4caf-a93a-1db4cd53c00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:29.160128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3708bb2-d565-4caf-a93a-1db4cd53c00d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:29.186528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 873/1772] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:29.187501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3708bb2-d565-4caf-a93a-1db4cd53c00d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:29.192654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c3708bb2-d565-4caf-a93a-1db4cd53c00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:29.251361 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3708bb2-d565-4caf-a93a-1db4cd53c00d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:29.256166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85f071-c1f4-40da-b0b3-2c8019216912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:29.256892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85f071-c1f4-40da-b0b3-2c8019216912 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:29.257238 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85f071-c1f4-40da-b0b3-2c8019216912 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:29.257524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85f071-c1f4-40da-b0b3-2c8019216912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:29.268130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85f071-c1f4-40da-b0b3-2c8019216912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:29.269639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85f071-c1f4-40da-b0b3-2c8019216912 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:29.298670 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 874/1773] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:29.300192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85f071-c1f4-40da-b0b3-2c8019216912 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:29.331564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85f071-c1f4-40da-b0b3-2c8019216912 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:29.341323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa536b46-d5e6-45d1-b4d2-eb9ea7e11aa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:29.342211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa536b46-d5e6-45d1-b4d2-eb9ea7e11aa4 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:29.342566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa536b46-d5e6-45d1-b4d2-eb9ea7e11aa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:29.342876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa536b46-d5e6-45d1-b4d2-eb9ea7e11aa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:29.363616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa536b46-d5e6-45d1-b4d2-eb9ea7e11aa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:29.363616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa536b46-d5e6-45d1-b4d2-eb9ea7e11aa4 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:29.387974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 875/1774] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:29.390624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa536b46-d5e6-45d1-b4d2-eb9ea7e11aa4 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:29.421230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa536b46-d5e6-45d1-b4d2-eb9ea7e11aa4 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:29.433333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c95f6-0ef4-4920-b8b5-7024927b94e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:29.434033 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c95f6-0ef4-4920-b8b5-7024927b94e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:29.434299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c95f6-0ef4-4920-b8b5-7024927b94e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:29.434587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c95f6-0ef4-4920-b8b5-7024927b94e5 admin admin] PATH_INFO: `/v3/projects/20db06d2caf84ad2957fbdf0acb7deb0/users/437f2e87a8574262a19a15c5034cc949/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:29.476801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c95f6-0ef4-4920-b8b5-7024927b94e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20db06d2caf84ad2957fbdf0acb7deb0, user_id=437f2e87a8574262a19a15c5034cc949, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:29.480479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c95f6-0ef4-4920-b8b5-7024927b94e5 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:29.503609 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6198f7e071e54f0f8c5186f5b031a7b4', 'request_id': 'req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:29.635596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6198f7e071e54f0f8c5186f5b031a7b4', 'request_id': 'req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:29.662056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 876/1775] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:29 2022] PUT /identity/v3/projects/20db06d2caf84ad2957fbdf0acb7deb0/users/437f2e87a8574262a19a15c5034cc949/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:29.662056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c95f6-0ef4-4920-b8b5-7024927b94e5 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:29.662483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6198f7e071e54f0f8c5186f5b031a7b4', 'request_id': 'req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:29.740766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] The token cache is being invalidate because project 6198f7e071e54f0f8c5186f5b031a7b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:29.740971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:29.758145 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:29.763913 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:29.770979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 900/1776] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:29 2022] DELETE /identity/v3/projects/6198f7e071e54f0f8c5186f5b031a7b4 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:29.770979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:29.775901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:29.848424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c95f6-0ef4-4920-b8b5-7024927b94e5 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:29.858001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acffc07-0a50-4b42-bce4-ce88a8b55819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:29.858001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acffc07-0a50-4b42-bce4-ce88a8b55819 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:29.858001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acffc07-0a50-4b42-bce4-ce88a8b55819 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:29.873143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acffc07-0a50-4b42-bce4-ce88a8b55819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:29.937305 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0acffc07-0a50-4b42-bce4-ce88a8b55819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:29.988562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab9a28-e28f-42bb-a0a7-5745b51c9540 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:30.004106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acffc07-0a50-4b42-bce4-ce88a8b55819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:30.006093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acffc07-0a50-4b42-bce4-ce88a8b55819 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:30.025307 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 877/1777] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:30.032055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acffc07-0a50-4b42-bce4-ce88a8b55819 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:30.034530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0acffc07-0a50-4b42-bce4-ce88a8b55819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:30.061405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acffc07-0a50-4b42-bce4-ce88a8b55819 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:30.076170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152b976-a46d-4a34-ad39-89edd7292d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:30.077403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152b976-a46d-4a34-ad39-89edd7292d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:30.079104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152b976-a46d-4a34-ad39-89edd7292d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:30.079373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152b976-a46d-4a34-ad39-89edd7292d3b admin admin] PATH_INFO: `/v3/projects/d50834ed551c4b589815a44354bd2870/users/f4857491584144d6bebc96706fbc7128/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:30.114935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183dc3a-2d6c-442b-9b4b-e5227ae6eb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:30.119164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183dc3a-2d6c-442b-9b4b-e5227ae6eb04 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:30.119530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152b976-a46d-4a34-ad39-89edd7292d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50834ed551c4b589815a44354bd2870, user_id=f4857491584144d6bebc96706fbc7128, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:30.122580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183dc3a-2d6c-442b-9b4b-e5227ae6eb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:30.122764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183dc3a-2d6c-442b-9b4b-e5227ae6eb04 admin admin] PATH_INFO: `/v3/projects/c72493932b9448e9a2379665e195f361/users/f2f51ff2addf4e1482a973f14db2d0e0/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:30.128830 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152b976-a46d-4a34-ad39-89edd7292d3b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:30.149798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183dc3a-2d6c-442b-9b4b-e5227ae6eb04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72493932b9448e9a2379665e195f361, user_id=f2f51ff2addf4e1482a973f14db2d0e0, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:30.153287 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183dc3a-2d6c-442b-9b4b-e5227ae6eb04 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:30.248045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 878/1778] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:30 2022] PUT /identity/v3/projects/d50834ed551c4b589815a44354bd2870/users/f4857491584144d6bebc96706fbc7128/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:30.259334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee2ba-7ee2-4ac6-a7cc-7d77478295ea None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:30.259334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee2ba-7ee2-4ac6-a7cc-7d77478295ea None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:30.259334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee2ba-7ee2-4ac6-a7cc-7d77478295ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:30.277655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 901/1779] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:29 2022] PUT /identity/v3/projects/c72493932b9448e9a2379665e195f361/users/f2f51ff2addf4e1482a973f14db2d0e0/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:30.285289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183dc3a-2d6c-442b-9b4b-e5227ae6eb04 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:30.289007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8183dc3a-2d6c-442b-9b4b-e5227ae6eb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:30.326777 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-d85ee2ba-7ee2-4ac6-a7cc-7d77478295ea None None] MFA Rules not processed for user `437f2e87a8574262a19a15c5034cc949`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:30.331465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d85ee2ba-7ee2-4ac6-a7cc-7d77478295ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:30.446796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183dc3a-2d6c-442b-9b4b-e5227ae6eb04 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:30.458882 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 879/1780] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:30.460305 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5949d-fba1-45b0-9df6-076316b85a23 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:30.460703 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5949d-fba1-45b0-9df6-076316b85a23 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:30.461047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5949d-fba1-45b0-9df6-076316b85a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:30.467666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:30.474945 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:30.474945 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:30.474945 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] PATH_INFO: `/v3/users/a75382812ffb48a892013e24eb5117d2` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:30.477369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=a75382812ffb48a892013e24eb5117d2)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:30.481316 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:30.510258 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-73a5949d-fba1-45b0-9df6-076316b85a23 None None] MFA Rules not processed for user `f4857491584144d6bebc96706fbc7128`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:30.511626 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-73a5949d-fba1-45b0-9df6-076316b85a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:30.536090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:30.639494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 880/1781] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:30.640451 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5949d-fba1-45b0-9df6-076316b85a23 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:30.663718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5949d-fba1-45b0-9df6-076316b85a23 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:30.669005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c95447-d2e1-4725-859c-f6036ec629b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:30.672361 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c95447-d2e1-4725-859c-f6036ec629b0 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:30.672361 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c95447-d2e1-4725-859c-f6036ec629b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:30.672361 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c95447-d2e1-4725-859c-f6036ec629b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:30.672361 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c95447-d2e1-4725-859c-f6036ec629b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:30.672831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c95447-d2e1-4725-859c-f6036ec629b0 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:30.693990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 881/1782] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:30.697099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c95447-d2e1-4725-859c-f6036ec629b0 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:30.701658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-12c95447-d2e1-4725-859c-f6036ec629b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:30.733481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c95447-d2e1-4725-859c-f6036ec629b0 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:30.764260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f941a1-c563-466a-9073-9813e1ca28e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:30.764260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f941a1-c563-466a-9073-9813e1ca28e9 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:30.764260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f941a1-c563-466a-9073-9813e1ca28e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:30.764260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f941a1-c563-466a-9073-9813e1ca28e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:30.764652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f941a1-c563-466a-9073-9813e1ca28e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:30.764652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f941a1-c563-466a-9073-9813e1ca28e9 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:31.039018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 882/1783] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:30 2022] POST /identity/v3/projects => generated 377 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:31.041896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f941a1-c563-466a-9073-9813e1ca28e9 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:31.044117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a75382812ffb48a892013e24eb5117d2'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:31.096462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a75382812ffb48a892013e24eb5117d2'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:31.098044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a75382812ffb48a892013e24eb5117d2'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:31.099281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a75382812ffb48a892013e24eb5117d2'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:31.220765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f941a1-c563-466a-9073-9813e1ca28e9 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:31.229140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e1f17-c11e-4609-bc8b-3a40d9f52a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:31.231079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e1f17-c11e-4609-bc8b-3a40d9f52a21 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:31.231558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e1f17-c11e-4609-bc8b-3a40d9f52a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:31.232567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e1f17-c11e-4609-bc8b-3a40d9f52a21 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:31.233370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e1f17-c11e-4609-bc8b-3a40d9f52a21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:31.236798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e1f17-c11e-4609-bc8b-3a40d9f52a21 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:31.367170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a75382812ffb48a892013e24eb5117d2'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:31.400592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 902/1784] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:30 2022] DELETE /identity/v3/users/a75382812ffb48a892013e24eb5117d2 => generated 0 bytes in 1122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:31.401454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:31.403024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:31.622329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e245e-0898-4947-9101-6119b48580ed admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:31.629079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97ef60-dd6d-41d7-b201-7678688c2b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:31.630064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97ef60-dd6d-41d7-b201-7678688c2b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:31.630284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97ef60-dd6d-41d7-b201-7678688c2b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:31.630800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97ef60-dd6d-41d7-b201-7678688c2b43 admin admin] PATH_INFO: `/v3/projects/c72493932b9448e9a2379665e195f361/users/f2f51ff2addf4e1482a973f14db2d0e0/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:31.637580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 883/1785] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:31 2022] POST /identity/v3/projects => generated 365 bytes in 598 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:31.641360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e1f17-c11e-4609-bc8b-3a40d9f52a21 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:31.643377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7a8e1f17-c11e-4609-bc8b-3a40d9f52a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:31.646222 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97ef60-dd6d-41d7-b201-7678688c2b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72493932b9448e9a2379665e195f361, user_id=f2f51ff2addf4e1482a973f14db2d0e0, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:31.650254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97ef60-dd6d-41d7-b201-7678688c2b43 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:31.664817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 903/1786] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:31 2022] PUT /identity/v3/projects/c72493932b9448e9a2379665e195f361/users/f2f51ff2addf4e1482a973f14db2d0e0/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:31.677166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31dc33-bc30-4970-9a1b-f239f9e0e61e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:31.677491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31dc33-bc30-4970-9a1b-f239f9e0e61e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:31.677764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31dc33-bc30-4970-9a1b-f239f9e0e61e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:31.679297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 904/1787] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 15:50:31 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:31.679830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31dc33-bc30-4970-9a1b-f239f9e0e61e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:31.682722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cb31dc33-bc30-4970-9a1b-f239f9e0e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:31.779251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e1f17-c11e-4609-bc8b-3a40d9f52a21 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:31.910330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31dc33-bc30-4970-9a1b-f239f9e0e61e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:31.917005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d3b29-b8fb-4ce2-8aee-8087c04224c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:31.917690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d3b29-b8fb-4ce2-8aee-8087c04224c7 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:31.917979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d3b29-b8fb-4ce2-8aee-8087c04224c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:31.918260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d3b29-b8fb-4ce2-8aee-8087c04224c7 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:31.918987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d3b29-b8fb-4ce2-8aee-8087c04224c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:31.921777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d3b29-b8fb-4ce2-8aee-8087c04224c7 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:31.957334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b3f77-fa51-42a3-bd74-8b42ee752dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:31.958116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b3f77-fa51-42a3-bd74-8b42ee752dce admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:31.958497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b3f77-fa51-42a3-bd74-8b42ee752dce admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:31.958869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b3f77-fa51-42a3-bd74-8b42ee752dce admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:31.959683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b3f77-fa51-42a3-bd74-8b42ee752dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:31.961924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b3f77-fa51-42a3-bd74-8b42ee752dce admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:32.480221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 884/1788] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:31 2022] POST /identity/v3/users => generated 295 bytes in 839 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:32.484933 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a99001-e0ea-4332-b3d8-da2ec49bd17f None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:32.484933 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a99001-e0ea-4332-b3d8-da2ec49bd17f None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:32.484933 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a99001-e0ea-4332-b3d8-da2ec49bd17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:32.489958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 905/1789] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:31 2022] POST /identity/v3/users => generated 289 bytes in 810 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:32.492048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777c9bd-276d-4ca1-b4bf-c78db1b8b6d9 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:32.492841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777c9bd-276d-4ca1-b4bf-c78db1b8b6d9 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:32.493286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777c9bd-276d-4ca1-b4bf-c78db1b8b6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:32.520298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e777c9bd-276d-4ca1-b4bf-c78db1b8b6d9 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:32.522437 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e777c9bd-276d-4ca1-b4bf-c78db1b8b6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:32.569325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-04a99001-e0ea-4332-b3d8-da2ec49bd17f None None] MFA Rules not processed for user `f2f51ff2addf4e1482a973f14db2d0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:32.570602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-04a99001-e0ea-4332-b3d8-da2ec49bd17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:32.674749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 906/1790] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 15:50:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:32.675600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777c9bd-276d-4ca1-b4bf-c78db1b8b6d9 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:32.687120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777c9bd-276d-4ca1-b4bf-c78db1b8b6d9 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:32.693294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57cdd-61d4-40ea-aa6d-1840e1c7c575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:32.694010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57cdd-61d4-40ea-aa6d-1840e1c7c575 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:32.694389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57cdd-61d4-40ea-aa6d-1840e1c7c575 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:32.694772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57cdd-61d4-40ea-aa6d-1840e1c7c575 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:32.695725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a57cdd-61d4-40ea-aa6d-1840e1c7c575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:32.698012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a57cdd-61d4-40ea-aa6d-1840e1c7c575 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:32.720391 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 907/1791] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:32.721396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57cdd-61d4-40ea-aa6d-1840e1c7c575 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:32.729956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 885/1792] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:32.729956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a99001-e0ea-4332-b3d8-da2ec49bd17f None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:32.731059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-04a99001-e0ea-4332-b3d8-da2ec49bd17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:32.745587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57cdd-61d4-40ea-aa6d-1840e1c7c575 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:32.750828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f32b5d-ad4d-47ba-b16d-0a1ce29f121f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:32.751813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f32b5d-ad4d-47ba-b16d-0a1ce29f121f admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:32.752129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a99001-e0ea-4332-b3d8-da2ec49bd17f None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:32.752629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f32b5d-ad4d-47ba-b16d-0a1ce29f121f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:32.753288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f32b5d-ad4d-47ba-b16d-0a1ce29f121f admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:32.754190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f32b5d-ad4d-47ba-b16d-0a1ce29f121f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:32.756674 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f32b5d-ad4d-47ba-b16d-0a1ce29f121f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:32.759186 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:32.760028 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:32.760406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:32.760797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] PATH_INFO: `/v3/projects/3b775594683d4bf197115b2dfa339c89` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:32.762145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b775594683d4bf197115b2dfa339c89)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:32.764813 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:32.794039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 908/1793] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:32.794812 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f32b5d-ad4d-47ba-b16d-0a1ce29f121f admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:32.802654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-23f32b5d-ad4d-47ba-b16d-0a1ce29f121f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:32.848052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f32b5d-ad4d-47ba-b16d-0a1ce29f121f admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:32.861586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9128acc-c5f2-4cf8-b2e4-ba87bea35a6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:32.862415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9128acc-c5f2-4cf8-b2e4-ba87bea35a6f service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:32.862834 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9128acc-c5f2-4cf8-b2e4-ba87bea35a6f service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:32.863182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9128acc-c5f2-4cf8-b2e4-ba87bea35a6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:32.874822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9128acc-c5f2-4cf8-b2e4-ba87bea35a6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:32.876828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9128acc-c5f2-4cf8-b2e4-ba87bea35a6f service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:32.892156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 909/1794] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:32.893612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9128acc-c5f2-4cf8-b2e4-ba87bea35a6f service nova] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:32.907963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9128acc-c5f2-4cf8-b2e4-ba87bea35a6f service nova] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:32.913971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c5124-4663-4d43-b6f6-a0a24a913ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:32.914825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c5124-4663-4d43-b6f6-a0a24a913ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:32.915373 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c5124-4663-4d43-b6f6-a0a24a913ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:32.916110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c5124-4663-4d43-b6f6-a0a24a913ec9 admin admin] PATH_INFO: `/v3/projects/d2688a801e35458db8753304b0929f47/users/4543a34b51bb485eba438ae7d69ac2cd/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:32.935836 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c5124-4663-4d43-b6f6-a0a24a913ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2688a801e35458db8753304b0929f47, user_id=4543a34b51bb485eba438ae7d69ac2cd, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:32.939369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c5124-4663-4d43-b6f6-a0a24a913ec9 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:33.379596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 910/1795] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:32 2022] PUT /identity/v3/projects/d2688a801e35458db8753304b0929f47/users/4543a34b51bb485eba438ae7d69ac2cd/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:33.380463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c5124-4663-4d43-b6f6-a0a24a913ec9 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:33.392742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-fd5c5124-4663-4d43-b6f6-a0a24a913ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:33.408078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b775594683d4bf197115b2dfa339c89', 'request_id': 'req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:33.420400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b775594683d4bf197115b2dfa339c89', 'request_id': 'req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:33.428909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b775594683d4bf197115b2dfa339c89', 'request_id': 'req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:33.477527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c5124-4663-4d43-b6f6-a0a24a913ec9 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:33.586823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] The token cache is being invalidate because project 3b775594683d4bf197115b2dfa339c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:33.593616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:33.593616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:33.593616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 886/1796] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:32 2022] DELETE /identity/v3/projects/3b775594683d4bf197115b2dfa339c89 => generated 0 bytes in 864 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:33.595696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:33.602642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:33.604125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73542d25-b828-40e7-b3e9-0ef4aae10f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:33.605363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73542d25-b828-40e7-b3e9-0ef4aae10f19 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:33.605750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73542d25-b828-40e7-b3e9-0ef4aae10f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:33.607781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73542d25-b828-40e7-b3e9-0ef4aae10f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:33.611738 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-73542d25-b828-40e7-b3e9-0ef4aae10f19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:33.713935 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12e49-4368-4b5a-9f22-9ae4142a0e1a admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:33.723204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:33.724898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:33.725499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:33.726008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] PATH_INFO: `/v3/projects/d0c8f1dcba8f46dfb7ce7ce9e099bbc3` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:33.727757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c8f1dcba8f46dfb7ce7ce9e099bbc3)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:33.731497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:33.746180 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73542d25-b828-40e7-b3e9-0ef4aae10f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:33.759649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73542d25-b828-40e7-b3e9-0ef4aae10f19 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:33.859981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 911/1797] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:33.859981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73542d25-b828-40e7-b3e9-0ef4aae10f19 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:33.859981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-73542d25-b828-40e7-b3e9-0ef4aae10f19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:33.859981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73542d25-b828-40e7-b3e9-0ef4aae10f19 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:33.875310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d246c9-fd10-4cc1-8f4f-140179fb4b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:33.879787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d246c9-fd10-4cc1-8f4f-140179fb4b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:33.883999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d246c9-fd10-4cc1-8f4f-140179fb4b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:33.887403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d246c9-fd10-4cc1-8f4f-140179fb4b0b admin admin] PATH_INFO: `/v3/projects/18b3c9ce84314cdf858155362075f336/users/fa5dbc82446d485e93e56b9eff6e0e55/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:33.961315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d246c9-fd10-4cc1-8f4f-140179fb4b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b3c9ce84314cdf858155362075f336, user_id=fa5dbc82446d485e93e56b9eff6e0e55, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:33.965534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d246c9-fd10-4cc1-8f4f-140179fb4b0b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:34.272078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 912/1798] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:33 2022] PUT /identity/v3/projects/18b3c9ce84314cdf858155362075f336/users/fa5dbc82446d485e93e56b9eff6e0e55/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:34.273967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca3946-7591-4040-9008-d4e3c165cfaa None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:34.274378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca3946-7591-4040-9008-d4e3c165cfaa None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:34.274731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca3946-7591-4040-9008-d4e3c165cfaa None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:34.276322 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 913/1799] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 15:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:34.280532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca3946-7591-4040-9008-d4e3c165cfaa None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:34.284206 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-72ca3946-7591-4040-9008-d4e3c165cfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:34.303404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c8f1dcba8f46dfb7ce7ce9e099bbc3', 'request_id': 'req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:34.317516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c8f1dcba8f46dfb7ce7ce9e099bbc3', 'request_id': 'req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:34.331982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c8f1dcba8f46dfb7ce7ce9e099bbc3', 'request_id': 'req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:34.401573 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca3946-7591-4040-9008-d4e3c165cfaa None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:34.407324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25848ebd-8851-4e67-84f7-736c616449f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:34.428194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25848ebd-8851-4e67-84f7-736c616449f9 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:34.428194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25848ebd-8851-4e67-84f7-736c616449f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:34.428194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25848ebd-8851-4e67-84f7-736c616449f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:34.428194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25848ebd-8851-4e67-84f7-736c616449f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:34.428194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25848ebd-8851-4e67-84f7-736c616449f9 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:34.428194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 914/1800] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:34.429035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517cb94-d8fe-4c2b-8849-b9ee24973b17 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:34.429035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517cb94-d8fe-4c2b-8849-b9ee24973b17 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:34.429035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517cb94-d8fe-4c2b-8849-b9ee24973b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:34.446281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-c517cb94-d8fe-4c2b-8849-b9ee24973b17 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:34.447568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c517cb94-d8fe-4c2b-8849-b9ee24973b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:34.468739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 915/1801] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:34.468739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517cb94-d8fe-4c2b-8849-b9ee24973b17 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:34.492750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517cb94-d8fe-4c2b-8849-b9ee24973b17 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:34.498425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbe97d-19f4-4353-b789-8fa593e5fb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:34.499048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbe97d-19f4-4353-b789-8fa593e5fb1e admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:34.499340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbe97d-19f4-4353-b789-8fa593e5fb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:34.499614 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbe97d-19f4-4353-b789-8fa593e5fb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:34.500304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbe97d-19f4-4353-b789-8fa593e5fb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:34.501922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbe97d-19f4-4353-b789-8fa593e5fb1e admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:34.514491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 916/1802] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:34.515918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4926a0-2f2c-4536-8700-4feae1a92301 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:34.516215 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4926a0-2f2c-4536-8700-4feae1a92301 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:34.516499 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4926a0-2f2c-4536-8700-4feae1a92301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:34.538971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-5e4926a0-2f2c-4536-8700-4feae1a92301 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:34.540306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5e4926a0-2f2c-4536-8700-4feae1a92301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:34.687228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 917/1803] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 15:50:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:34.688461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4926a0-2f2c-4536-8700-4feae1a92301 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:34.711372 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4926a0-2f2c-4536-8700-4feae1a92301 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:34.721326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9de0f2-7b6d-4671-a441-026227b607f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:34.722537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9de0f2-7b6d-4671-a441-026227b607f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:34.723154 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9de0f2-7b6d-4671-a441-026227b607f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:34.723603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9de0f2-7b6d-4671-a441-026227b607f8 admin admin] PATH_INFO: `/v3/projects/d2688a801e35458db8753304b0929f47/users/4543a34b51bb485eba438ae7d69ac2cd/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:34.727967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9de0f2-7b6d-4671-a441-026227b607f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2688a801e35458db8753304b0929f47, user_id=4543a34b51bb485eba438ae7d69ac2cd, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:34.733531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9de0f2-7b6d-4671-a441-026227b607f8 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:34.755666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 918/1804] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:34 2022] PUT /identity/v3/projects/d2688a801e35458db8753304b0929f47/users/4543a34b51bb485eba438ae7d69ac2cd/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:34.757400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9309be-66d3-46ee-9ad5-23b79a36ce92 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:34.757702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9309be-66d3-46ee-9ad5-23b79a36ce92 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:34.758018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9309be-66d3-46ee-9ad5-23b79a36ce92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:34.797535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-5f9309be-66d3-46ee-9ad5-23b79a36ce92 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:34.799837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5f9309be-66d3-46ee-9ad5-23b79a36ce92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:34.882800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 919/1805] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:34.886461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9309be-66d3-46ee-9ad5-23b79a36ce92 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:34.886578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] The token cache is being invalidate because project d0c8f1dcba8f46dfb7ce7ce9e099bbc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:34.886578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:34.888572 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:34.890915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 887/1806] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:33 2022] DELETE /identity/v3/projects/d0c8f1dcba8f46dfb7ce7ce9e099bbc3 => generated 0 bytes in 1297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:34.895807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:34.899092 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9309be-66d3-46ee-9ad5-23b79a36ce92 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:34.899391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:34.913718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1a1c8-0410-4522-966b-a15399e15048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:34.914403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1a1c8-0410-4522-966b-a15399e15048 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:34.914808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1a1c8-0410-4522-966b-a15399e15048 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:34.915094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1a1c8-0410-4522-966b-a15399e15048 admin admin] PATH_INFO: `/v3/projects/18b3c9ce84314cdf858155362075f336/users/fa5dbc82446d485e93e56b9eff6e0e55/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:34.917310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1a1c8-0410-4522-966b-a15399e15048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b3c9ce84314cdf858155362075f336, user_id=fa5dbc82446d485e93e56b9eff6e0e55, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:34.921201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1a1c8-0410-4522-966b-a15399e15048 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:34.953528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 920/1807] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:34 2022] PUT /identity/v3/projects/18b3c9ce84314cdf858155362075f336/users/fa5dbc82446d485e93e56b9eff6e0e55/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:34.953528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bec6b9-eabd-47e2-9fff-6e29e0aee2e0 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:34.953528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bec6b9-eabd-47e2-9fff-6e29e0aee2e0 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:34.953528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bec6b9-eabd-47e2-9fff-6e29e0aee2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:34.991787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-84bec6b9-eabd-47e2-9fff-6e29e0aee2e0 None None] MFA Rules not processed for user `4543a34b51bb485eba438ae7d69ac2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:34.993173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-84bec6b9-eabd-47e2-9fff-6e29e0aee2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:35.044295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2671e81-e4f9-4d5e-b5c6-65bac0808fb8 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:35.120157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110652c9-717c-4cba-962e-54b8cf39975d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:35.120605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110652c9-717c-4cba-962e-54b8cf39975d service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:35.120896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110652c9-717c-4cba-962e-54b8cf39975d service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:35.121175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110652c9-717c-4cba-962e-54b8cf39975d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:35.123648 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-110652c9-717c-4cba-962e-54b8cf39975d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:35.144133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110652c9-717c-4cba-962e-54b8cf39975d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:35.145951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110652c9-717c-4cba-962e-54b8cf39975d service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:35.162994 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 888/1808] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:35.165004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110652c9-717c-4cba-962e-54b8cf39975d service nova] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:35.168155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-110652c9-717c-4cba-962e-54b8cf39975d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:35.176276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 921/1809] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:35.177126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ce731-b6c0-4150-b0c0-5a394509ed57 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:35.177415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ce731-b6c0-4150-b0c0-5a394509ed57 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:35.177698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ce731-b6c0-4150-b0c0-5a394509ed57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:35.212028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-075ce731-b6c0-4150-b0c0-5a394509ed57 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:35.213367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-075ce731-b6c0-4150-b0c0-5a394509ed57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:35.281311 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110652c9-717c-4cba-962e-54b8cf39975d service nova] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:35.289931 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3081569-9d71-4582-b17a-e7d6954ecfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:35.292576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3081569-9d71-4582-b17a-e7d6954ecfe0 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:35.292576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3081569-9d71-4582-b17a-e7d6954ecfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:35.292576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3081569-9d71-4582-b17a-e7d6954ecfe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:35.294535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3081569-9d71-4582-b17a-e7d6954ecfe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:35.295819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3081569-9d71-4582-b17a-e7d6954ecfe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:35.298871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3081569-9d71-4582-b17a-e7d6954ecfe0 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:35.351120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 922/1810] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:35.353041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414a781-fa2f-47c5-b0e8-33f126b724f0 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:35.353689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414a781-fa2f-47c5-b0e8-33f126b724f0 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:35.354260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414a781-fa2f-47c5-b0e8-33f126b724f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:35.371167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 889/1811] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:35.383821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3081569-9d71-4582-b17a-e7d6954ecfe0 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:35.389052 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a3081569-9d71-4582-b17a-e7d6954ecfe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:35.408162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1414a781-fa2f-47c5-b0e8-33f126b724f0 None None] MFA Rules not processed for user `fa5dbc82446d485e93e56b9eff6e0e55`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:35.409425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1414a781-fa2f-47c5-b0e8-33f126b724f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:35.554835 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3081569-9d71-4582-b17a-e7d6954ecfe0 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:35.561610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 923/1812] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:35.563397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889554bc-3c7d-4f29-a38f-9974bb98af86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:35.564508 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889554bc-3c7d-4f29-a38f-9974bb98af86 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:35.564799 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889554bc-3c7d-4f29-a38f-9974bb98af86 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:35.565064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889554bc-3c7d-4f29-a38f-9974bb98af86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:35.565992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69cb4d-eb8b-487c-8edd-433b60258e06 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:35.566283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69cb4d-eb8b-487c-8edd-433b60258e06 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:35.566559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69cb4d-eb8b-487c-8edd-433b60258e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:35.569507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-889554bc-3c7d-4f29-a38f-9974bb98af86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:35.591330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-4a69cb4d-eb8b-487c-8edd-433b60258e06 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:35.592678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4a69cb4d-eb8b-487c-8edd-433b60258e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:35.608953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 924/1813] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:35.608953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69cb4d-eb8b-487c-8edd-433b60258e06 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:35.631908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69cb4d-eb8b-487c-8edd-433b60258e06 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:35.640013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a6362-205a-491c-8345-7b7d05759b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:35.641053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a6362-205a-491c-8345-7b7d05759b52 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:35.641053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a6362-205a-491c-8345-7b7d05759b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:35.641662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a6362-205a-491c-8345-7b7d05759b52 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:35.642664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a6362-205a-491c-8345-7b7d05759b52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:35.644731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a6362-205a-491c-8345-7b7d05759b52 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:35.698361 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889554bc-3c7d-4f29-a38f-9974bb98af86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:35.699903 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889554bc-3c7d-4f29-a38f-9974bb98af86 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:35.711963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 890/1814] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:35.712810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889554bc-3c7d-4f29-a38f-9974bb98af86 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:35.728495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889554bc-3c7d-4f29-a38f-9974bb98af86 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:35.736616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954631d1-b678-481c-8a0c-e67910212b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:35.737531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954631d1-b678-481c-8a0c-e67910212b62 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:35.737922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954631d1-b678-481c-8a0c-e67910212b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:35.738354 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954631d1-b678-481c-8a0c-e67910212b62 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:35.739549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954631d1-b678-481c-8a0c-e67910212b62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:35.739910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954631d1-b678-481c-8a0c-e67910212b62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:35.741960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954631d1-b678-481c-8a0c-e67910212b62 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:35.763010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 891/1815] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:35.768435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954631d1-b678-481c-8a0c-e67910212b62 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:35.809422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954631d1-b678-481c-8a0c-e67910212b62 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:35.816372 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c43d0-1608-48d6-abdf-e38ba174921e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:35.817742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c43d0-1608-48d6-abdf-e38ba174921e admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:35.818024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c43d0-1608-48d6-abdf-e38ba174921e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:35.818310 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c43d0-1608-48d6-abdf-e38ba174921e admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:35.818943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c43d0-1608-48d6-abdf-e38ba174921e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:35.821688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c43d0-1608-48d6-abdf-e38ba174921e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:35.967115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 892/1816] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:35 2022] POST /identity/v3/projects => generated 403 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:35.970818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 925/1817] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:35 2022] POST /identity/v3/projects => generated 357 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:35.971742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a6362-205a-491c-8345-7b7d05759b52 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:35.971892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c43d0-1608-48d6-abdf-e38ba174921e admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:36.070880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c43d0-1608-48d6-abdf-e38ba174921e admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:36.077174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60016943-7119-430a-a5d9-641635166698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:36.077972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60016943-7119-430a-a5d9-641635166698 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:36.078409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60016943-7119-430a-a5d9-641635166698 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:36.078898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60016943-7119-430a-a5d9-641635166698 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:36.079824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60016943-7119-430a-a5d9-641635166698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:36.081887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60016943-7119-430a-a5d9-641635166698 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:36.083829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a6362-205a-491c-8345-7b7d05759b52 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:36.091772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d70b7e-1a16-4604-a55d-178678a96170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:36.092930 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d70b7e-1a16-4604-a55d-178678a96170 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:36.093362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d70b7e-1a16-4604-a55d-178678a96170 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:36.093794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d70b7e-1a16-4604-a55d-178678a96170 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:36.094600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d70b7e-1a16-4604-a55d-178678a96170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:36.096663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d70b7e-1a16-4604-a55d-178678a96170 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:36.410145 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:36.420299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 926/1818] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:35 2022] POST /identity/v3/users => generated 302 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:36.430616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 893/1819] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:35 2022] POST /identity/v3/users => generated 279 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:36.430806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d70b7e-1a16-4604-a55d-178678a96170 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:36.430916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60016943-7119-430a-a5d9-641635166698 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:36.438852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d70b7e-1a16-4604-a55d-178678a96170 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:36.439117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60016943-7119-430a-a5d9-641635166698 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:36.444212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7a8d2-c6fe-463b-905f-c91c8ddc3393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:36.445156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7a8d2-c6fe-463b-905f-c91c8ddc3393 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:36.449054 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f390c-5c28-4bbb-a40b-7b2fb1c270e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:36.449706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f390c-5c28-4bbb-a40b-7b2fb1c270e4 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:36.450422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f390c-5c28-4bbb-a40b-7b2fb1c270e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:36.450422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f390c-5c28-4bbb-a40b-7b2fb1c270e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:36.451712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7a8d2-c6fe-463b-905f-c91c8ddc3393 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:36.451849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f390c-5c28-4bbb-a40b-7b2fb1c270e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:36.452239 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7a8d2-c6fe-463b-905f-c91c8ddc3393 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:36.453474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f390c-5c28-4bbb-a40b-7b2fb1c270e4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:36.454163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7a8d2-c6fe-463b-905f-c91c8ddc3393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:36.457328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7a8d2-c6fe-463b-905f-c91c8ddc3393 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:36.469561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 894/1820] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:50:36.470557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f390c-5c28-4bbb-a40b-7b2fb1c270e4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:36.483131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 927/1821] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:36.484516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7a8d2-c6fe-463b-905f-c91c8ddc3393 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:36.499430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f390c-5c28-4bbb-a40b-7b2fb1c270e4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:36.505560 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7118717-0a25-4e5c-b57c-c6ef042f749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:36.506362 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7118717-0a25-4e5c-b57c-c6ef042f749b admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:36.506628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7118717-0a25-4e5c-b57c-c6ef042f749b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:36.506901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7118717-0a25-4e5c-b57c-c6ef042f749b admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:36.507561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7118717-0a25-4e5c-b57c-c6ef042f749b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:36.509216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7118717-0a25-4e5c-b57c-c6ef042f749b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:36.518381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7a8d2-c6fe-463b-905f-c91c8ddc3393 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:36.521018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 895/1822] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:36.526117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7118717-0a25-4e5c-b57c-c6ef042f749b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:36.528563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0f8f2-147f-4da7-bb72-3888a5c50d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:36.529934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0f8f2-147f-4da7-bb72-3888a5c50d17 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:36.530558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0f8f2-147f-4da7-bb72-3888a5c50d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:36.530997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0f8f2-147f-4da7-bb72-3888a5c50d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:36.531987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0f8f2-147f-4da7-bb72-3888a5c50d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:36.533854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0f8f2-147f-4da7-bb72-3888a5c50d17 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:36.537409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7118717-0a25-4e5c-b57c-c6ef042f749b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:36.550082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7769f6-9088-4e0d-af06-444b4ea6e2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:36.550082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7769f6-9088-4e0d-af06-444b4ea6e2e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:36.550082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7769f6-9088-4e0d-af06-444b4ea6e2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:36.550082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7769f6-9088-4e0d-af06-444b4ea6e2e9 admin admin] PATH_INFO: `/v3/projects/659f383d58c240efb3e15789e4cf8942/users/62c9133e061841aea7f20e00dacab5ef/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:36.550483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 928/1823] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:36.553070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0f8f2-147f-4da7-bb72-3888a5c50d17 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:36.563846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7769f6-9088-4e0d-af06-444b4ea6e2e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=659f383d58c240efb3e15789e4cf8942, user_id=62c9133e061841aea7f20e00dacab5ef, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:36.565072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0f8f2-147f-4da7-bb72-3888a5c50d17 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:36.566776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7769f6-9088-4e0d-af06-444b4ea6e2e9 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:36.571741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74107d4-8616-413d-b97c-78cc935cd463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:36.572189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74107d4-8616-413d-b97c-78cc935cd463 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:36.572648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74107d4-8616-413d-b97c-78cc935cd463 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:36.573063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74107d4-8616-413d-b97c-78cc935cd463 admin admin] PATH_INFO: `/v3/projects/8eeddae577c6416db592f2386d4f6982/users/80221c7dddbc4e3f809d04e6f8c42ef1/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:36.601123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74107d4-8616-413d-b97c-78cc935cd463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eeddae577c6416db592f2386d4f6982, user_id=80221c7dddbc4e3f809d04e6f8c42ef1, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:36.604031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74107d4-8616-413d-b97c-78cc935cd463 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:36.802980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 929/1824] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:36 2022] PUT /identity/v3/projects/8eeddae577c6416db592f2386d4f6982/users/80221c7dddbc4e3f809d04e6f8c42ef1/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:50:36.809423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 896/1825] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:36 2022] PUT /identity/v3/projects/659f383d58c240efb3e15789e4cf8942/users/62c9133e061841aea7f20e00dacab5ef/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:36.811049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d012d-f6be-4b5f-9130-57a453bfd9d2 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:36.811566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d012d-f6be-4b5f-9130-57a453bfd9d2 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:36.812089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d012d-f6be-4b5f-9130-57a453bfd9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:36.839849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faca97-a12c-407d-9d06-a426161b2f33 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:36.843330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faca97-a12c-407d-9d06-a426161b2f33 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:36.843846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faca97-a12c-407d-9d06-a426161b2f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:36.886298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-c00d012d-f6be-4b5f-9130-57a453bfd9d2 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:36.887927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c00d012d-f6be-4b5f-9130-57a453bfd9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:36.915080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-42faca97-a12c-407d-9d06-a426161b2f33 None None] MFA Rules not processed for user `62c9133e061841aea7f20e00dacab5ef`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:36.915080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-42faca97-a12c-407d-9d06-a426161b2f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:37.004006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 897/1826] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:37.004006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42faca97-a12c-407d-9d06-a426161b2f33 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:37.004006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-42faca97-a12c-407d-9d06-a426161b2f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:37.006383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 930/1827] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:37.008040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d012d-f6be-4b5f-9130-57a453bfd9d2 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:37.025940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d012d-f6be-4b5f-9130-57a453bfd9d2 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:37.032008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bc56a-24af-4fc0-9c6a-6df964c72237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:37.032851 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bc56a-24af-4fc0-9c6a-6df964c72237 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:37.033563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bc56a-24af-4fc0-9c6a-6df964c72237 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:37.034009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bc56a-24af-4fc0-9c6a-6df964c72237 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:37.034797 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9bc56a-24af-4fc0-9c6a-6df964c72237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:37.037711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9bc56a-24af-4fc0-9c6a-6df964c72237 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:37.107955 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42faca97-a12c-407d-9d06-a426161b2f33 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:37.116974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58ae8b-4b61-4036-818c-b821112dad2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:37.118452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58ae8b-4b61-4036-818c-b821112dad2c service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:37.119223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58ae8b-4b61-4036-818c-b821112dad2c service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:37.120244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58ae8b-4b61-4036-818c-b821112dad2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:37.261899 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58ae8b-4b61-4036-818c-b821112dad2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:37.264042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58ae8b-4b61-4036-818c-b821112dad2c service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:37.266065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 931/1828] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:37 2022] POST /identity/v3/projects => generated 355 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:37.268125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bc56a-24af-4fc0-9c6a-6df964c72237 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:37.271371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9c9bc56a-24af-4fc0-9c6a-6df964c72237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:37.276516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 898/1829] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:37.277640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58ae8b-4b61-4036-818c-b821112dad2c service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:37.290692 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58ae8b-4b61-4036-818c-b821112dad2c service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:37.402566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25275b3b-3b3c-457c-ac58-2f9780f8d05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:37.403211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25275b3b-3b3c-457c-ac58-2f9780f8d05b admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:37.405758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25275b3b-3b3c-457c-ac58-2f9780f8d05b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:37.405758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25275b3b-3b3c-457c-ac58-2f9780f8d05b admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:37.406494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25275b3b-3b3c-457c-ac58-2f9780f8d05b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:37.425859 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25275b3b-3b3c-457c-ac58-2f9780f8d05b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:37.443630 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bc56a-24af-4fc0-9c6a-6df964c72237 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:37.450038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fb82f-2189-46fa-b6d7-819133c4ad17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:37.451441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb82f-2189-46fa-b6d7-819133c4ad17 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:37.451746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb82f-2189-46fa-b6d7-819133c4ad17 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:37.452078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb82f-2189-46fa-b6d7-819133c4ad17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:37.477211 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fb82f-2189-46fa-b6d7-819133c4ad17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:37.478701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fb82f-2189-46fa-b6d7-819133c4ad17 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:37.515251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 932/1830] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:37.525839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fb82f-2189-46fa-b6d7-819133c4ad17 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:37.526903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-487fb82f-2189-46fa-b6d7-819133c4ad17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:37.627748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 899/1831] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:37 2022] POST /identity/v3/users => generated 284 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:37.628939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25275b3b-3b3c-457c-ac58-2f9780f8d05b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:37.642738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25275b3b-3b3c-457c-ac58-2f9780f8d05b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:37.648155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fb82f-2189-46fa-b6d7-819133c4ad17 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:37.648288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f5f80-e51f-41c4-9572-d824bcd054f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:37.649057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f5f80-e51f-41c4-9572-d824bcd054f4 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:37.649333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f5f80-e51f-41c4-9572-d824bcd054f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:37.649731 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f5f80-e51f-41c4-9572-d824bcd054f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:37.650525 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f5f80-e51f-41c4-9572-d824bcd054f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:37.652333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f5f80-e51f-41c4-9572-d824bcd054f4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:37.659770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64243a83-2af1-45dd-88dd-fba28ea93bb8 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f51ff2addf4e1482a973f14db2d0e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c72493932b9448e9a2379665e195f361', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:37.660979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64243a83-2af1-45dd-88dd-fba28ea93bb8 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:37.661570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64243a83-2af1-45dd-88dd-fba28ea93bb8 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:37.661815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64243a83-2af1-45dd-88dd-fba28ea93bb8 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:37.662456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64243a83-2af1-45dd-88dd-fba28ea93bb8 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:37.662846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64243a83-2af1-45dd-88dd-fba28ea93bb8 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:37.665620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64243a83-2af1-45dd-88dd-fba28ea93bb8 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:37.671495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 900/1832] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:37.671495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f5f80-e51f-41c4-9572-d824bcd054f4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:37.698570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 933/1833] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:37.700449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64243a83-2af1-45dd-88dd-fba28ea93bb8 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:37.715544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64243a83-2af1-45dd-88dd-fba28ea93bb8 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:37.733709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edd427-1b4e-4ad1-90bb-fa0f98a04977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:37.733709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edd427-1b4e-4ad1-90bb-fa0f98a04977 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:37.733709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edd427-1b4e-4ad1-90bb-fa0f98a04977 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:37.733709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edd427-1b4e-4ad1-90bb-fa0f98a04977 admin admin] PATH_INFO: `/v3/projects/d2a1d075a06349158c04679a85241acf/users/5072aaa7c9494340ac8e253dbf605fdd/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:37.938981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edd427-1b4e-4ad1-90bb-fa0f98a04977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a1d075a06349158c04679a85241acf, user_id=5072aaa7c9494340ac8e253dbf605fdd, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:37.938981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edd427-1b4e-4ad1-90bb-fa0f98a04977 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:37.940084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f5f80-e51f-41c4-9572-d824bcd054f4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:37.940084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a9d58-7816-4de3-b620-1b6962ae3101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:37.940084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a9d58-7816-4de3-b620-1b6962ae3101 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:37.940084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a9d58-7816-4de3-b620-1b6962ae3101 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:37.940725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a9d58-7816-4de3-b620-1b6962ae3101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:37.940725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d10a9d58-7816-4de3-b620-1b6962ae3101 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:38.005938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a9d58-7816-4de3-b620-1b6962ae3101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:38.007937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a9d58-7816-4de3-b620-1b6962ae3101 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:38.023356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 901/1834] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:38.024607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a9d58-7816-4de3-b620-1b6962ae3101 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:38.040199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a9d58-7816-4de3-b620-1b6962ae3101 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:38.048116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b5f02-1d22-42c8-b957-b6e9ab0681b0 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f51ff2addf4e1482a973f14db2d0e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c72493932b9448e9a2379665e195f361', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:38.048891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b5f02-1d22-42c8-b957-b6e9ab0681b0 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:38.049207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b5f02-1d22-42c8-b957-b6e9ab0681b0 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:38.051658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b5f02-1d22-42c8-b957-b6e9ab0681b0 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:38.052353 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b5f02-1d22-42c8-b957-b6e9ab0681b0 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:38.055223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b5f02-1d22-42c8-b957-b6e9ab0681b0 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:38.296166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 934/1835] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:37 2022] PUT /identity/v3/projects/d2a1d075a06349158c04679a85241acf/users/5072aaa7c9494340ac8e253dbf605fdd/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:38.301933 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edd427-1b4e-4ad1-90bb-fa0f98a04977 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:38.307575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 902/1836] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:38 2022] POST /identity/v3/projects => generated 354 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:38.309385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b5f02-1d22-42c8-b957-b6e9ab0681b0 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:38.385837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edd427-1b4e-4ad1-90bb-fa0f98a04977 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:38.401857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b5f02-1d22-42c8-b957-b6e9ab0681b0 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:38.407742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e46c01-fea3-4144-ab35-e91b7ed39344 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f51ff2addf4e1482a973f14db2d0e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c72493932b9448e9a2379665e195f361', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:38.408528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e46c01-fea3-4144-ab35-e91b7ed39344 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:38.408921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e46c01-fea3-4144-ab35-e91b7ed39344 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:38.409306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e46c01-fea3-4144-ab35-e91b7ed39344 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:38.410068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e46c01-fea3-4144-ab35-e91b7ed39344 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:38.410545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e46c01-fea3-4144-ab35-e91b7ed39344 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:38.412287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e46c01-fea3-4144-ab35-e91b7ed39344 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:38.424352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 903/1837] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:38.453516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e944478-d4d1-4bef-b97d-7f43834fbb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:38.454304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e944478-d4d1-4bef-b97d-7f43834fbb79 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:38.454594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e944478-d4d1-4bef-b97d-7f43834fbb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:38.454876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e944478-d4d1-4bef-b97d-7f43834fbb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:38.455543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e944478-d4d1-4bef-b97d-7f43834fbb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:38.458244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e944478-d4d1-4bef-b97d-7f43834fbb79 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:38.479976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 935/1838] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:50:38.480253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e46c01-fea3-4144-ab35-e91b7ed39344 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:38.481055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e944478-d4d1-4bef-b97d-7f43834fbb79 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:38.494420 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e46c01-fea3-4144-ab35-e91b7ed39344 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:38.502031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09892475-8e56-4dd4-bd8b-fdb639d14880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:38.502834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09892475-8e56-4dd4-bd8b-fdb639d14880 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:38.505886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09892475-8e56-4dd4-bd8b-fdb639d14880 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:38.505886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09892475-8e56-4dd4-bd8b-fdb639d14880 admin admin] PATH_INFO: `/v3/projects/d2a1d075a06349158c04679a85241acf/users/5072aaa7c9494340ac8e253dbf605fdd/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:38.506109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e944478-d4d1-4bef-b97d-7f43834fbb79 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:38.506290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09892475-8e56-4dd4-bd8b-fdb639d14880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a1d075a06349158c04679a85241acf, user_id=5072aaa7c9494340ac8e253dbf605fdd, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:38.510565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09892475-8e56-4dd4-bd8b-fdb639d14880 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:38.518771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f51ff2addf4e1482a973f14db2d0e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c72493932b9448e9a2379665e195f361', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:38.520240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:38.520702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:38.521138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] PATH_INFO: `/v3/projects/39a7221a8a8048c3870b2e257fe7c5cc` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:38.522545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: Authorizing `identity:delete_project(project_id=39a7221a8a8048c3870b2e257fe7c5cc)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:38.525613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:38.532623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 904/1839] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:38 2022] PUT /identity/v3/projects/d2a1d075a06349158c04679a85241acf/users/5072aaa7c9494340ac8e253dbf605fdd/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:50:38.542605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580205e-9d71-4cd4-8e9c-169011e65745 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:38.543165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580205e-9d71-4cd4-8e9c-169011e65745 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:38.543572 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580205e-9d71-4cd4-8e9c-169011e65745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:38.608913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-a580205e-9d71-4cd4-8e9c-169011e65745 None None] MFA Rules not processed for user `5072aaa7c9494340ac8e253dbf605fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:38.610380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a580205e-9d71-4cd4-8e9c-169011e65745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:38.698260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 905/1840] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:38 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:38.712325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580205e-9d71-4cd4-8e9c-169011e65745 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:38.791920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580205e-9d71-4cd4-8e9c-169011e65745 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:38.792351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39a7221a8a8048c3870b2e257fe7c5cc', 'request_id': 'req-2d4ac691-7125-413a-9289-fe93648d9491'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:38.863069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc7bf2-87cc-4d24-9fa7-4031c4d8778c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:38.866398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc7bf2-87cc-4d24-9fa7-4031c4d8778c service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:38.866398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc7bf2-87cc-4d24-9fa7-4031c4d8778c service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:38.866398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:38.866398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc7bf2-87cc-4d24-9fa7-4031c4d8778c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:38.932360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc7bf2-87cc-4d24-9fa7-4031c4d8778c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:38.935014 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc7bf2-87cc-4d24-9fa7-4031c4d8778c service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:38.944383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 906/1841] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:50:38.954904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39a7221a8a8048c3870b2e257fe7c5cc', 'request_id': 'req-2d4ac691-7125-413a-9289-fe93648d9491'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:38.962613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39a7221a8a8048c3870b2e257fe7c5cc', 'request_id': 'req-2d4ac691-7125-413a-9289-fe93648d9491'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:38.973605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] The token cache is being invalidate because project 39a7221a8a8048c3870b2e257fe7c5cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:38.973937 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:38.974588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:38.976192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 936/1842] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:38 2022] DELETE /identity/v3/projects/39a7221a8a8048c3870b2e257fe7c5cc => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:39.296531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc7bf2-87cc-4d24-9fa7-4031c4d8778c service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:39.306308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc7bf2-87cc-4d24-9fa7-4031c4d8778c service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:39.311337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abb99f-1be7-4020-9326-17f8d957f145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:39.312145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abb99f-1be7-4020-9326-17f8d957f145 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:39.312534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abb99f-1be7-4020-9326-17f8d957f145 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:39.312925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abb99f-1be7-4020-9326-17f8d957f145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:39.314943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-85abb99f-1be7-4020-9326-17f8d957f145 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:39.381793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abb99f-1be7-4020-9326-17f8d957f145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:39.383332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abb99f-1be7-4020-9326-17f8d957f145 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:39.394913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 907/1843] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:39 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:39.852454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:39.854056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:39.899722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:39.910102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e4e3-ddae-4f0a-8aaa-8c6e1a911fc8 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:39.910414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e4e3-ddae-4f0a-8aaa-8c6e1a911fc8 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:39.910704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e4e3-ddae-4f0a-8aaa-8c6e1a911fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:39.944801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ac691-7125-413a-9289-fe93648d9491 tempest-VolumeQuotaClassesTest-2012871562 tempest-VolumeQuotaClassesTest-2012871562-project-admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:39.950294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef376d7d-fb23-430c-8ce0-32355c9cd867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:39.951712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376d7d-fb23-430c-8ce0-32355c9cd867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:39.952198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376d7d-fb23-430c-8ce0-32355c9cd867 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:39.952724 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376d7d-fb23-430c-8ce0-32355c9cd867 admin admin] PATH_INFO: `/v3/users/3008ae15fffc4ae1b353ce4fe2de9bfc` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:39.954185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef376d7d-fb23-430c-8ce0-32355c9cd867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3008ae15fffc4ae1b353ce4fe2de9bfc)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:39.956486 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef376d7d-fb23-430c-8ce0-32355c9cd867 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:39.958608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-cec2e4e3-ddae-4f0a-8aaa-8c6e1a911fc8 None None] MFA Rules not processed for user `80221c7dddbc4e3f809d04e6f8c42ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:39.960234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cec2e4e3-ddae-4f0a-8aaa-8c6e1a911fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:40.042466 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 908/1844] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:40.057626 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2e4e3-ddae-4f0a-8aaa-8c6e1a911fc8 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:40.059282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cec2e4e3-ddae-4f0a-8aaa-8c6e1a911fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:40.079658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2e4e3-ddae-4f0a-8aaa-8c6e1a911fc8 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:40.086054 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749555cb-4b16-4497-8933-1bc9970ed6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:40.087201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749555cb-4b16-4497-8933-1bc9970ed6bf admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:40.095159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749555cb-4b16-4497-8933-1bc9970ed6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:40.095159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749555cb-4b16-4497-8933-1bc9970ed6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:40.095159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749555cb-4b16-4497-8933-1bc9970ed6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:40.095159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749555cb-4b16-4497-8933-1bc9970ed6bf admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:40.272621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 909/1845] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:40 2022] POST /identity/v3/projects => generated 403 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:40.277446 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749555cb-4b16-4497-8933-1bc9970ed6bf admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:40.278543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ef376d7d-fb23-430c-8ce0-32355c9cd867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3008ae15fffc4ae1b353ce4fe2de9bfc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:40.293340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ef376d7d-fb23-430c-8ce0-32355c9cd867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3008ae15fffc4ae1b353ce4fe2de9bfc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:40.294102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ef376d7d-fb23-430c-8ce0-32355c9cd867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3008ae15fffc4ae1b353ce4fe2de9bfc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:40.294778 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ef376d7d-fb23-430c-8ce0-32355c9cd867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3008ae15fffc4ae1b353ce4fe2de9bfc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:40.345278 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749555cb-4b16-4497-8933-1bc9970ed6bf admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:40.350016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8772e0-891a-4ce8-a10a-230bad9c75c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:40.350626 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8772e0-891a-4ce8-a10a-230bad9c75c6 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:40.350912 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8772e0-891a-4ce8-a10a-230bad9c75c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:40.351191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8772e0-891a-4ce8-a10a-230bad9c75c6 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:40.351822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8772e0-891a-4ce8-a10a-230bad9c75c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:40.353840 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8772e0-891a-4ce8-a10a-230bad9c75c6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:40.386171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ef376d7d-fb23-430c-8ce0-32355c9cd867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3008ae15fffc4ae1b353ce4fe2de9bfc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:40.401246 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 937/1846] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:39 2022] DELETE /identity/v3/users/3008ae15fffc4ae1b353ce4fe2de9bfc => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:40.407510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff8823-a39f-4c62-888d-5b524166cf6c None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:40.407939 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff8823-a39f-4c62-888d-5b524166cf6c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:40.408390 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff8823-a39f-4c62-888d-5b524166cf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:40.428549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-90ff8823-a39f-4c62-888d-5b524166cf6c None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:40.429766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-90ff8823-a39f-4c62-888d-5b524166cf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:40.455575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 910/1847] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:40 2022] POST /identity/v3/users => generated 308 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:50:40.459394 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8772e0-891a-4ce8-a10a-230bad9c75c6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:40.461614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-be8772e0-891a-4ce8-a10a-230bad9c75c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:40.475438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8772e0-891a-4ce8-a10a-230bad9c75c6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:40.500555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 938/1848] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:40.507435 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff8823-a39f-4c62-888d-5b524166cf6c None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:40.511072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-90ff8823-a39f-4c62-888d-5b524166cf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:40.539161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcff861-97ee-4294-a5f0-f3eee0ae7388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:40.539913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcff861-97ee-4294-a5f0-f3eee0ae7388 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:40.540124 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcff861-97ee-4294-a5f0-f3eee0ae7388 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:40.540423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcff861-97ee-4294-a5f0-f3eee0ae7388 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:40.541093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcff861-97ee-4294-a5f0-f3eee0ae7388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:40.542788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcff861-97ee-4294-a5f0-f3eee0ae7388 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:40.555972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 911/1849] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:40.557758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcff861-97ee-4294-a5f0-f3eee0ae7388 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:40.577401 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcff861-97ee-4294-a5f0-f3eee0ae7388 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:40.585424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff8823-a39f-4c62-888d-5b524166cf6c None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:40.585424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d2bcd-c1cb-4e91-89e0-3b72fd733f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:40.586046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d2bcd-c1cb-4e91-89e0-3b72fd733f5e service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:40.586359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d2bcd-c1cb-4e91-89e0-3b72fd733f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:40.586596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d2bcd-c1cb-4e91-89e0-3b72fd733f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:40.587135 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee69513-28eb-47d3-aec8-d32bdbca363a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:40.588096 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee69513-28eb-47d3-aec8-d32bdbca363a admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:40.588816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee69513-28eb-47d3-aec8-d32bdbca363a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:40.589199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee69513-28eb-47d3-aec8-d32bdbca363a admin admin] PATH_INFO: `/v3/projects/4bbabe98240741e3a51710e54cae353b/users/f763c8a2b5fa47fbbe7afa508c2b2a96/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:40.601122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d2bcd-c1cb-4e91-89e0-3b72fd733f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:40.602657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d2bcd-c1cb-4e91-89e0-3b72fd733f5e service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:40.615474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 939/1850] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:40.615887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee69513-28eb-47d3-aec8-d32bdbca363a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bbabe98240741e3a51710e54cae353b, user_id=f763c8a2b5fa47fbbe7afa508c2b2a96, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:40.618714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee69513-28eb-47d3-aec8-d32bdbca363a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:40.803527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 912/1851] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:40 2022] PUT /identity/v3/projects/4bbabe98240741e3a51710e54cae353b/users/f763c8a2b5fa47fbbe7afa508c2b2a96/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:50:40.807962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d2bcd-c1cb-4e91-89e0-3b72fd733f5e service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:40.818848 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d2bcd-c1cb-4e91-89e0-3b72fd733f5e service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:40.891330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d7ad7-737e-4a88-bd0c-49519be75ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:40.891330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d7ad7-737e-4a88-bd0c-49519be75ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:40.891991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d7ad7-737e-4a88-bd0c-49519be75ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:40.891991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d7ad7-737e-4a88-bd0c-49519be75ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:40.892721 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d7ad7-737e-4a88-bd0c-49519be75ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:40.894689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d7ad7-737e-4a88-bd0c-49519be75ba9 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:40.906756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 940/1852] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:40.911209 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee69513-28eb-47d3-aec8-d32bdbca363a admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:40.921590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee69513-28eb-47d3-aec8-d32bdbca363a admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:40.926241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcc96b-a5e4-41b7-aa73-be20259f0ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:40.926898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcc96b-a5e4-41b7-aa73-be20259f0ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:40.927219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcc96b-a5e4-41b7-aa73-be20259f0ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:40.932391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcc96b-a5e4-41b7-aa73-be20259f0ad1 admin admin] PATH_INFO: `/v3/projects/4bbabe98240741e3a51710e54cae353b/users/f763c8a2b5fa47fbbe7afa508c2b2a96/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:40.932391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcc96b-a5e4-41b7-aa73-be20259f0ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bbabe98240741e3a51710e54cae353b, user_id=f763c8a2b5fa47fbbe7afa508c2b2a96, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:40.932600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcc96b-a5e4-41b7-aa73-be20259f0ad1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:40.942891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 913/1853] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:40 2022] PUT /identity/v3/projects/4bbabe98240741e3a51710e54cae353b/users/f763c8a2b5fa47fbbe7afa508c2b2a96/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:41.117615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:41.289412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d7ad7-737e-4a88-bd0c-49519be75ba9 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:41.291245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-648d7ad7-737e-4a88-bd0c-49519be75ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:41.304171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d7ad7-737e-4a88-bd0c-49519be75ba9 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:41.374796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:41.375431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:41.375918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:41.376031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] PATH_INFO: `/v3/projects/af0fa4f49c3743f8b3c2b8f901997b8c` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:41.377164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=af0fa4f49c3743f8b3c2b8f901997b8c)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:41.379098 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:41.510219 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af0fa4f49c3743f8b3c2b8f901997b8c', 'request_id': 'req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:41.653631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af0fa4f49c3743f8b3c2b8f901997b8c', 'request_id': 'req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:41.659183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af0fa4f49c3743f8b3c2b8f901997b8c', 'request_id': 'req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:41.673255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] The token cache is being invalidate because project af0fa4f49c3743f8b3c2b8f901997b8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:41.673643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:41.674405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:41.677367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 941/1854] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:41 2022] DELETE /identity/v3/projects/af0fa4f49c3743f8b3c2b8f901997b8c => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:41.690783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcc96b-a5e4-41b7-aa73-be20259f0ad1 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:41.692803 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-81dcc96b-a5e4-41b7-aa73-be20259f0ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:41.709762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcc96b-a5e4-41b7-aa73-be20259f0ad1 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:41.784317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a1994-d22c-4c38-bba2-d4f09446db9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:41.784317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a1994-d22c-4c38-bba2-d4f09446db9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:41.784317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a1994-d22c-4c38-bba2-d4f09446db9e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:41.784317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a1994-d22c-4c38-bba2-d4f09446db9e admin admin] PATH_INFO: `/v3/users/f2f51ff2addf4e1482a973f14db2d0e0` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:41.784674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a1994-d22c-4c38-bba2-d4f09446db9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2f51ff2addf4e1482a973f14db2d0e0)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:41.784674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a1994-d22c-4c38-bba2-d4f09446db9e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:42.132679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6a1a1994-d22c-4c38-bba2-d4f09446db9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2f51ff2addf4e1482a973f14db2d0e0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:42.133345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6a1a1994-d22c-4c38-bba2-d4f09446db9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2f51ff2addf4e1482a973f14db2d0e0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:42.146522 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6a1a1994-d22c-4c38-bba2-d4f09446db9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2f51ff2addf4e1482a973f14db2d0e0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:42.229480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6a1a1994-d22c-4c38-bba2-d4f09446db9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2f51ff2addf4e1482a973f14db2d0e0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:42.240684 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6a1a1994-d22c-4c38-bba2-d4f09446db9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2f51ff2addf4e1482a973f14db2d0e0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:42.246651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 914/1855] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:41 2022] DELETE /identity/v3/users/f2f51ff2addf4e1482a973f14db2d0e0 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:42.812641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:42.815794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8ec63-a99d-4378-8693-9695a3c14a6e None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:42.816666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8ec63-a99d-4378-8693-9695a3c14a6e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:42.817727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8ec63-a99d-4378-8693-9695a3c14a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:42.824761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f46cc-c05b-4ec9-b5d1-1491b9805b6a admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:42.888083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-93e8ec63-a99d-4378-8693-9695a3c14a6e None None] MFA Rules not processed for user `f763c8a2b5fa47fbbe7afa508c2b2a96`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:42.888083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-93e8ec63-a99d-4378-8693-9695a3c14a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:42.898240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:42.899006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:42.899420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:42.899803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] PATH_INFO: `/v3/projects/c72493932b9448e9a2379665e195f361` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:42.901742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c72493932b9448e9a2379665e195f361)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:42.903689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:42.944613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 915/1856] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:42 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:43.132417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c72493932b9448e9a2379665e195f361', 'request_id': 'req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:43.231390 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c72493932b9448e9a2379665e195f361', 'request_id': 'req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:43.238239 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c72493932b9448e9a2379665e195f361', 'request_id': 'req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:43.249079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] The token cache is being invalidate because project c72493932b9448e9a2379665e195f361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:43.249375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:43.250105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4e659f2c-e020-40cb-ac03-b1a26bfa7fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:43.251639 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 942/1857] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:42 2022] DELETE /identity/v3/projects/c72493932b9448e9a2379665e195f361 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:43.256923 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052521c-849a-4c02-a255-5816b8315b73 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:43.257213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052521c-849a-4c02-a255-5816b8315b73 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:43.257492 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052521c-849a-4c02-a255-5816b8315b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:43.277351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-4052521c-849a-4c02-a255-5816b8315b73 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:43.278620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4052521c-849a-4c02-a255-5816b8315b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:43.338582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 916/1858] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:43.347568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d9b33-03d4-480f-a3e9-3fdc97c347e7 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:43.347959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d9b33-03d4-480f-a3e9-3fdc97c347e7 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:43.348256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d9b33-03d4-480f-a3e9-3fdc97c347e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:43.369029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-197d9b33-03d4-480f-a3e9-3fdc97c347e7 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:43.370276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-197d9b33-03d4-480f-a3e9-3fdc97c347e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:43.380179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 943/1859] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:43.384569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052521c-849a-4c02-a255-5816b8315b73 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:43.395272 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052521c-849a-4c02-a255-5816b8315b73 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:43.400043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8514b59-5522-42b3-b441-0075add5cd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:43.400656 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8514b59-5522-42b3-b441-0075add5cd1b admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:43.400951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8514b59-5522-42b3-b441-0075add5cd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:43.401228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8514b59-5522-42b3-b441-0075add5cd1b admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:43.403278 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8514b59-5522-42b3-b441-0075add5cd1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:43.403708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8514b59-5522-42b3-b441-0075add5cd1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:43.405955 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8514b59-5522-42b3-b441-0075add5cd1b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:43.417966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 917/1860] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:43.422292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d9b33-03d4-480f-a3e9-3fdc97c347e7 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:43.431841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d9b33-03d4-480f-a3e9-3fdc97c347e7 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:43.437112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07690324-cc19-434c-9af4-51e9889f39bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:43.437727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07690324-cc19-434c-9af4-51e9889f39bf admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:43.438008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07690324-cc19-434c-9af4-51e9889f39bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:43.438288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07690324-cc19-434c-9af4-51e9889f39bf admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:43.438940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07690324-cc19-434c-9af4-51e9889f39bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:43.440894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07690324-cc19-434c-9af4-51e9889f39bf admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:43.568575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 944/1861] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:43 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:43.572761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8514b59-5522-42b3-b441-0075add5cd1b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:43.583708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8514b59-5522-42b3-b441-0075add5cd1b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:43.652818 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c449d-f060-455b-99a3-ce2e11cc7533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:43.653436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c449d-f060-455b-99a3-ce2e11cc7533 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:43.653726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c449d-f060-455b-99a3-ce2e11cc7533 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:43.654052 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c449d-f060-455b-99a3-ce2e11cc7533 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:43.654698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c449d-f060-455b-99a3-ce2e11cc7533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:43.656750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c449d-f060-455b-99a3-ce2e11cc7533 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:43.803494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 918/1862] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:43 2022] POST /identity/v3/users => generated 284 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:43.808897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07690324-cc19-434c-9af4-51e9889f39bf admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:43.822693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07690324-cc19-434c-9af4-51e9889f39bf admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:43.828807 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8f98d-6d24-4df4-9cfe-a62880f1ec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:43.829508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8f98d-6d24-4df4-9cfe-a62880f1ec51 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:43.829860 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8f98d-6d24-4df4-9cfe-a62880f1ec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:43.830197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8f98d-6d24-4df4-9cfe-a62880f1ec51 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:43.831055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8f98d-6d24-4df4-9cfe-a62880f1ec51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:43.832869 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8f98d-6d24-4df4-9cfe-a62880f1ec51 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:43.844605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 945/1863] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:50:43.849208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c449d-f060-455b-99a3-ce2e11cc7533 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:43.861653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c449d-f060-455b-99a3-ce2e11cc7533 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:43.863388 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca18c-f265-41a5-bf8d-341748501c6e None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:43.863750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca18c-f265-41a5-bf8d-341748501c6e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:43.864232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca18c-f265-41a5-bf8d-341748501c6e None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:43.866795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 946/1864] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 15:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:43.870941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a468e-4c1d-4bed-89db-6bdcab715a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:43.871772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a468e-4c1d-4bed-89db-6bdcab715a11 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:43.872094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a468e-4c1d-4bed-89db-6bdcab715a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:43.872370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a468e-4c1d-4bed-89db-6bdcab715a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:43.873054 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a468e-4c1d-4bed-89db-6bdcab715a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:43.874716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a468e-4c1d-4bed-89db-6bdcab715a11 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:43.880453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b54ec9-ab03-4087-8bba-4e0c5f7cb009 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:43.881693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b54ec9-ab03-4087-8bba-4e0c5f7cb009 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:43.882233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b54ec9-ab03-4087-8bba-4e0c5f7cb009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:43.890412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 919/1865] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:50:43.891454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a468e-4c1d-4bed-89db-6bdcab715a11 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:43.902172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a468e-4c1d-4bed-89db-6bdcab715a11 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:43.907936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe562c-748e-4a67-8fa8-b90ff8637cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:43.908677 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe562c-748e-4a67-8fa8-b90ff8637cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:43.908962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe562c-748e-4a67-8fa8-b90ff8637cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:43.909247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe562c-748e-4a67-8fa8-b90ff8637cfc admin admin] PATH_INFO: `/v3/projects/e371cf1572db4921b33fb6f4d586d660/users/78f3db4069f3434ea1ab4553ab6de815/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:43.912301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-17b54ec9-ab03-4087-8bba-4e0c5f7cb009 None None] MFA Rules not processed for user `b28f987c90df4c19a12bb1819dc65cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:43.914264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-17b54ec9-ab03-4087-8bba-4e0c5f7cb009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:43.926698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe562c-748e-4a67-8fa8-b90ff8637cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e371cf1572db4921b33fb6f4d586d660, user_id=78f3db4069f3434ea1ab4553ab6de815, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:43.929753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe562c-748e-4a67-8fa8-b90ff8637cfc admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:43.997371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 947/1866] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 15:50:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:44.010349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b54ec9-ab03-4087-8bba-4e0c5f7cb009 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:44.012864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-17b54ec9-ab03-4087-8bba-4e0c5f7cb009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:44.027746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b54ec9-ab03-4087-8bba-4e0c5f7cb009 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:44.033666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721a1ce-d6d5-4112-8f36-ab9236e213a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:44.034317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721a1ce-d6d5-4112-8f36-ab9236e213a3 service placement] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:44.034609 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721a1ce-d6d5-4112-8f36-ab9236e213a3 service placement] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:44.034905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721a1ce-d6d5-4112-8f36-ab9236e213a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:44.045013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721a1ce-d6d5-4112-8f36-ab9236e213a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:44.046886 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721a1ce-d6d5-4112-8f36-ab9236e213a3 service placement] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:44.066438 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 948/1867] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:50:44.071756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 920/1868] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:43 2022] PUT /identity/v3/projects/e371cf1572db4921b33fb6f4d586d660/users/78f3db4069f3434ea1ab4553ab6de815/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:44.083733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8325ee-951c-455d-bd09-40fb2167d8fa None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:44.084181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8325ee-951c-455d-bd09-40fb2167d8fa None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:44.089627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8325ee-951c-455d-bd09-40fb2167d8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:44.125591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9f8325ee-951c-455d-bd09-40fb2167d8fa None None] MFA Rules not processed for user `78f3db4069f3434ea1ab4553ab6de815`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:44.127066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9f8325ee-951c-455d-bd09-40fb2167d8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:44.264662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 949/1869] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:44.269207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe562c-748e-4a67-8fa8-b90ff8637cfc admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:44.424486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe562c-748e-4a67-8fa8-b90ff8637cfc admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:44.429987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2abd91-6e9a-4a38-b3f4-ef076d69ebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:44.430920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2abd91-6e9a-4a38-b3f4-ef076d69ebfc admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:44.431215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2abd91-6e9a-4a38-b3f4-ef076d69ebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:44.431738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2abd91-6e9a-4a38-b3f4-ef076d69ebfc admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:44.432562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2abd91-6e9a-4a38-b3f4-ef076d69ebfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:44.435044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2abd91-6e9a-4a38-b3f4-ef076d69ebfc admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:45.193662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:45.200295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 921/1870] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:44 2022] POST /identity/v3/projects => generated 365 bytes in 930 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:45.203151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8325ee-951c-455d-bd09-40fb2167d8fa None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:45.232954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8325ee-951c-455d-bd09-40fb2167d8fa None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:45.339237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244969ce-005d-43b0-8c8b-487bfa367299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:45.340174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244969ce-005d-43b0-8c8b-487bfa367299 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:45.340689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244969ce-005d-43b0-8c8b-487bfa367299 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:45.340899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244969ce-005d-43b0-8c8b-487bfa367299 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:45.342339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244969ce-005d-43b0-8c8b-487bfa367299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:45.345539 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244969ce-005d-43b0-8c8b-487bfa367299 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:45.482644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 950/1871] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:45 2022] POST /identity/v3/users => generated 289 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:45.482962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2abd91-6e9a-4a38-b3f4-ef076d69ebfc admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:45.501180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2abd91-6e9a-4a38-b3f4-ef076d69ebfc admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:45.509424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20637ca0-b266-4fde-9ab1-c92da37663a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:45.510708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20637ca0-b266-4fde-9ab1-c92da37663a1 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:45.510995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20637ca0-b266-4fde-9ab1-c92da37663a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:45.511279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20637ca0-b266-4fde-9ab1-c92da37663a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:45.512076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20637ca0-b266-4fde-9ab1-c92da37663a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:45.514391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20637ca0-b266-4fde-9ab1-c92da37663a1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:45.534340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 922/1872] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:50:45.536220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244969ce-005d-43b0-8c8b-487bfa367299 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:45.550273 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244969ce-005d-43b0-8c8b-487bfa367299 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:45.556633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388b68d-ec84-4fea-aba9-9fd242ff112f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:45.558025 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b68d-ec84-4fea-aba9-9fd242ff112f admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:45.558795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b68d-ec84-4fea-aba9-9fd242ff112f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:45.559537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b68d-ec84-4fea-aba9-9fd242ff112f admin admin] PATH_INFO: `/v3/projects/aa0e6435408540d38ba71d5a436ee25e/users/2f5d561a54224953844be84c371f5cd4/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:45.589844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388b68d-ec84-4fea-aba9-9fd242ff112f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0e6435408540d38ba71d5a436ee25e, user_id=2f5d561a54224953844be84c371f5cd4, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:45.593708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388b68d-ec84-4fea-aba9-9fd242ff112f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:45.812068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 951/1873] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:45 2022] PUT /identity/v3/projects/aa0e6435408540d38ba71d5a436ee25e/users/2f5d561a54224953844be84c371f5cd4/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:45.815191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20637ca0-b266-4fde-9ab1-c92da37663a1 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:45.825198 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20637ca0-b266-4fde-9ab1-c92da37663a1 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:45.845901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5549100-346f-48ce-ad97-e574e66b8acf None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:45.846227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5549100-346f-48ce-ad97-e574e66b8acf None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:45.846472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5549100-346f-48ce-ad97-e574e66b8acf None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:45.848114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 952/1874] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 15:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:45.858087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081eeff-c0d9-4824-a2d4-975e52e133fb None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:45.858392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081eeff-c0d9-4824-a2d4-975e52e133fb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:45.858683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081eeff-c0d9-4824-a2d4-975e52e133fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:45.885462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-d081eeff-c0d9-4824-a2d4-975e52e133fb None None] MFA Rules not processed for user `b28f987c90df4c19a12bb1819dc65cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:45.886759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-d081eeff-c0d9-4824-a2d4-975e52e133fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:45.911255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597c0ee-ab1f-474b-82d7-670cfa5da43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:45.911890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597c0ee-ab1f-474b-82d7-670cfa5da43d admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:45.912205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597c0ee-ab1f-474b-82d7-670cfa5da43d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:45.912486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597c0ee-ab1f-474b-82d7-670cfa5da43d admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:45.913140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597c0ee-ab1f-474b-82d7-670cfa5da43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:45.914808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597c0ee-ab1f-474b-82d7-670cfa5da43d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:45.927905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 923/1875] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:50:45.930250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597c0ee-ab1f-474b-82d7-670cfa5da43d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:45.941745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597c0ee-ab1f-474b-82d7-670cfa5da43d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:45.946447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddc83b-a226-4c33-b590-249c38c273cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:45.947121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddc83b-a226-4c33-b590-249c38c273cc admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:45.947455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddc83b-a226-4c33-b590-249c38c273cc admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:45.947741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddc83b-a226-4c33-b590-249c38c273cc admin admin] PATH_INFO: `/v3/projects/aa0e6435408540d38ba71d5a436ee25e/users/2f5d561a54224953844be84c371f5cd4/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:45.949754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddc83b-a226-4c33-b590-249c38c273cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0e6435408540d38ba71d5a436ee25e, user_id=2f5d561a54224953844be84c371f5cd4, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:45.952470 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddc83b-a226-4c33-b590-249c38c273cc admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:45.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 924/1876] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:45 2022] PUT /identity/v3/projects/aa0e6435408540d38ba71d5a436ee25e/users/2f5d561a54224953844be84c371f5cd4/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:50:45.968263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 953/1877] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 15:50:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:45.978520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041de02d-e585-4128-b7cd-8b1917ca0d5f None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:45.978520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041de02d-e585-4128-b7cd-8b1917ca0d5f None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:45.978520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041de02d-e585-4128-b7cd-8b1917ca0d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:45.981398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081eeff-c0d9-4824-a2d4-975e52e133fb None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:45.993235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081eeff-c0d9-4824-a2d4-975e52e133fb None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:46.009502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-041de02d-e585-4128-b7cd-8b1917ca0d5f None None] MFA Rules not processed for user `2f5d561a54224953844be84c371f5cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:46.011130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-041de02d-e585-4128-b7cd-8b1917ca0d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:46.100318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 925/1878] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:46.100318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041de02d-e585-4128-b7cd-8b1917ca0d5f None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:46.102063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-041de02d-e585-4128-b7cd-8b1917ca0d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:46.119904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b189-d7de-4043-bcee-d3a9050a4c72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:46.120957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b189-d7de-4043-bcee-d3a9050a4c72 service placement] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:46.121591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b189-d7de-4043-bcee-d3a9050a4c72 service placement] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:46.122243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b189-d7de-4043-bcee-d3a9050a4c72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:46.133715 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3b189-d7de-4043-bcee-d3a9050a4c72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:46.136384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3b189-d7de-4043-bcee-d3a9050a4c72 service placement] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:46.188702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 954/1879] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:50:46.230456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041de02d-e585-4128-b7cd-8b1917ca0d5f None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:46.235850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:46.239309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:46.239309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:46.239309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] PATH_INFO: `/v3/users/80221c7dddbc4e3f809d04e6f8c42ef1` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:46.239309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80221c7dddbc4e3f809d04e6f8c42ef1)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:46.241710 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:46.283787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b189-d7de-4043-bcee-d3a9050a4c72 service placement] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:46.285992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ede3b189-d7de-4043-bcee-d3a9050a4c72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:46.383803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:46.440366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b189-d7de-4043-bcee-d3a9050a4c72 service placement] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:46.440366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c1a98-0c73-4b44-ac4c-d8f9c87d8f05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:46.440366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c1a98-0c73-4b44-ac4c-d8f9c87d8f05 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:46.440366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c1a98-0c73-4b44-ac4c-d8f9c87d8f05 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:46.440864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c1a98-0c73-4b44-ac4c-d8f9c87d8f05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:46.440864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cf7c1a98-0c73-4b44-ac4c-d8f9c87d8f05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:46.518999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c1a98-0c73-4b44-ac4c-d8f9c87d8f05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:46.520687 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c1a98-0c73-4b44-ac4c-d8f9c87d8f05 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:46.537134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 955/1880] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:50:46.540108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c1a98-0c73-4b44-ac4c-d8f9c87d8f05 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:46.544222 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cf7c1a98-0c73-4b44-ac4c-d8f9c87d8f05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:46.547020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80221c7dddbc4e3f809d04e6f8c42ef1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:46.548529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80221c7dddbc4e3f809d04e6f8c42ef1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:46.567261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c1a98-0c73-4b44-ac4c-d8f9c87d8f05 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:46.572738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80221c7dddbc4e3f809d04e6f8c42ef1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:46.581645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a97064-72c1-4155-89c9-17cb9dc97836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:46.582279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a97064-72c1-4155-89c9-17cb9dc97836 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:46.582568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a97064-72c1-4155-89c9-17cb9dc97836 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:46.582850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a97064-72c1-4155-89c9-17cb9dc97836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:46.594900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a97064-72c1-4155-89c9-17cb9dc97836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:46.596472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a97064-72c1-4155-89c9-17cb9dc97836 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:46.610086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 956/1881] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:46.707376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80221c7dddbc4e3f809d04e6f8c42ef1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:46.720478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80221c7dddbc4e3f809d04e6f8c42ef1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:46.758767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 926/1882] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:46 2022] DELETE /identity/v3/users/80221c7dddbc4e3f809d04e6f8c42ef1 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:47.026817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a97064-72c1-4155-89c9-17cb9dc97836 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:47.035826 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a97064-72c1-4155-89c9-17cb9dc97836 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:47.086074 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40630779-ca05-4804-ba20-3fa26edfe9ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:47.086619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630779-ca05-4804-ba20-3fa26edfe9ad service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:47.086889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630779-ca05-4804-ba20-3fa26edfe9ad service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:47.087288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630779-ca05-4804-ba20-3fa26edfe9ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:47.089058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-40630779-ca05-4804-ba20-3fa26edfe9ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:47.136111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:47.137778 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:47.149361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40630779-ca05-4804-ba20-3fa26edfe9ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:47.150817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40630779-ca05-4804-ba20-3fa26edfe9ad service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:47.163053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 957/1883] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:47.232029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a553e0-3e46-4cdf-8225-6cffa5c8c957 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:47.238389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:47.239113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:47.239405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:47.239686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] PATH_INFO: `/v3/projects/8eeddae577c6416db592f2386d4f6982` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:47.240892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eeddae577c6416db592f2386d4f6982)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:47.242858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:47.363941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eeddae577c6416db592f2386d4f6982', 'request_id': 'req-871cdfb6-401f-4c51-bac4-c25555110015'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:47.369392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eeddae577c6416db592f2386d4f6982', 'request_id': 'req-871cdfb6-401f-4c51-bac4-c25555110015'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:47.376271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eeddae577c6416db592f2386d4f6982', 'request_id': 'req-871cdfb6-401f-4c51-bac4-c25555110015'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:47.509604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] The token cache is being invalidate because project 8eeddae577c6416db592f2386d4f6982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:47.509861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:47.511608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:47.513502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 927/1884] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:47 2022] DELETE /identity/v3/projects/8eeddae577c6416db592f2386d4f6982 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:50:47.517644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40630779-ca05-4804-ba20-3fa26edfe9ad service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:47.535037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40630779-ca05-4804-ba20-3fa26edfe9ad service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:47.537702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:47.612851 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5ed5c-3d2f-4e9a-9866-a970f646f4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:47.613509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5ed5c-3d2f-4e9a-9866-a970f646f4d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:47.614265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5ed5c-3d2f-4e9a-9866-a970f646f4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:47.614265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5ed5c-3d2f-4e9a-9866-a970f646f4d7 admin admin] PATH_INFO: `/v3/users/f763c8a2b5fa47fbbe7afa508c2b2a96` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:47.615226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5ed5c-3d2f-4e9a-9866-a970f646f4d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f763c8a2b5fa47fbbe7afa508c2b2a96)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:47.617331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5ed5c-3d2f-4e9a-9866-a970f646f4d7 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:47.869369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-07a5ed5c-3d2f-4e9a-9866-a970f646f4d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f763c8a2b5fa47fbbe7afa508c2b2a96'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:47.883319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-07a5ed5c-3d2f-4e9a-9866-a970f646f4d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f763c8a2b5fa47fbbe7afa508c2b2a96'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:47.884313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-07a5ed5c-3d2f-4e9a-9866-a970f646f4d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f763c8a2b5fa47fbbe7afa508c2b2a96'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:47.885071 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-07a5ed5c-3d2f-4e9a-9866-a970f646f4d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f763c8a2b5fa47fbbe7afa508c2b2a96'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:47.977754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-07a5ed5c-3d2f-4e9a-9866-a970f646f4d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f763c8a2b5fa47fbbe7afa508c2b2a96'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:47.991480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 958/1885] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:47 2022] DELETE /identity/v3/users/f763c8a2b5fa47fbbe7afa508c2b2a96 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:50:48.276077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:48.277562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:48.293387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cdfb6-401f-4c51-bac4-c25555110015 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:48.349927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:48.350568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:48.350850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:48.351127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] PATH_INFO: `/v3/projects/4bbabe98240741e3a51710e54cae353b` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:48.352326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bbabe98240741e3a51710e54cae353b)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:48.354271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:48.466207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bbabe98240741e3a51710e54cae353b', 'request_id': 'req-8220140b-b44a-4100-bfc0-74a5e8eff5df'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:48.471738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bbabe98240741e3a51710e54cae353b', 'request_id': 'req-8220140b-b44a-4100-bfc0-74a5e8eff5df'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:48.477873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bbabe98240741e3a51710e54cae353b', 'request_id': 'req-8220140b-b44a-4100-bfc0-74a5e8eff5df'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:48.566137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] The token cache is being invalidate because project 4bbabe98240741e3a51710e54cae353b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:48.566432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:48.567089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8220140b-b44a-4100-bfc0-74a5e8eff5df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:48.568700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 928/1886] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:48 2022] DELETE /identity/v3/projects/4bbabe98240741e3a51710e54cae353b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:48.573536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef64df-1e4c-462a-95f4-74828bfef8d6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:48.574017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef64df-1e4c-462a-95f4-74828bfef8d6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:48.574128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef64df-1e4c-462a-95f4-74828bfef8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:48.581075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:48.594828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-61ef64df-1e4c-462a-95f4-74828bfef8d6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:48.596076 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-61ef64df-1e4c-462a-95f4-74828bfef8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:48.656996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 959/1887] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:48.664459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c968f70-e6db-4fe3-bc5c-12a00304a6a3 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:48.664764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c968f70-e6db-4fe3-bc5c-12a00304a6a3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:48.665237 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c968f70-e6db-4fe3-bc5c-12a00304a6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:48.693533 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7c968f70-e6db-4fe3-bc5c-12a00304a6a3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:48.695241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7c968f70-e6db-4fe3-bc5c-12a00304a6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:48.707563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 929/1888] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:48.712707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef64df-1e4c-462a-95f4-74828bfef8d6 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:48.725156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef64df-1e4c-462a-95f4-74828bfef8d6 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:48.730370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4276828-2d92-4886-bc73-250482c438c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:48.730995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276828-2d92-4886-bc73-250482c438c4 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:48.731282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276828-2d92-4886-bc73-250482c438c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:48.731562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276828-2d92-4886-bc73-250482c438c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:48.732541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4276828-2d92-4886-bc73-250482c438c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:48.732795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4276828-2d92-4886-bc73-250482c438c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:48.734604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4276828-2d92-4886-bc73-250482c438c4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:48.747636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 960/1889] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:48.752435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c968f70-e6db-4fe3-bc5c-12a00304a6a3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:48.764500 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c968f70-e6db-4fe3-bc5c-12a00304a6a3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:48.770344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78701da-0577-4fb0-ad3e-e875dc1903fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:48.771038 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78701da-0577-4fb0-ad3e-e875dc1903fe admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:48.773190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78701da-0577-4fb0-ad3e-e875dc1903fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:48.773190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78701da-0577-4fb0-ad3e-e875dc1903fe admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:48.773190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78701da-0577-4fb0-ad3e-e875dc1903fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:48.774585 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78701da-0577-4fb0-ad3e-e875dc1903fe admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:48.865124 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 930/1890] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:48 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:48.869318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4276828-2d92-4886-bc73-250482c438c4 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:48.879967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4276828-2d92-4886-bc73-250482c438c4 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:48.938433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddacfc5-b069-4492-b7a7-2d524e457524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:48.939041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddacfc5-b069-4492-b7a7-2d524e457524 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:48.939324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddacfc5-b069-4492-b7a7-2d524e457524 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:48.939597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddacfc5-b069-4492-b7a7-2d524e457524 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:48.940250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddacfc5-b069-4492-b7a7-2d524e457524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:48.942212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddacfc5-b069-4492-b7a7-2d524e457524 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:49.078924 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 961/1891] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:48 2022] POST /identity/v3/users => generated 295 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:49.083185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78701da-0577-4fb0-ad3e-e875dc1903fe admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:49.094706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78701da-0577-4fb0-ad3e-e875dc1903fe admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:49.099536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957fd69-5165-47b1-83e6-e1dcc2b5217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:49.100209 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957fd69-5165-47b1-83e6-e1dcc2b5217f admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:49.100498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957fd69-5165-47b1-83e6-e1dcc2b5217f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:49.100778 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957fd69-5165-47b1-83e6-e1dcc2b5217f admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:49.101439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957fd69-5165-47b1-83e6-e1dcc2b5217f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:49.103039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957fd69-5165-47b1-83e6-e1dcc2b5217f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:49.116067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 931/1892] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:49.118493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddacfc5-b069-4492-b7a7-2d524e457524 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:49.130603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddacfc5-b069-4492-b7a7-2d524e457524 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:49.135550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a69e15-4925-4959-a4b3-e46292c0f960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:49.136241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a69e15-4925-4959-a4b3-e46292c0f960 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:49.136522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a69e15-4925-4959-a4b3-e46292c0f960 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:49.136795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a69e15-4925-4959-a4b3-e46292c0f960 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:49.137460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a69e15-4925-4959-a4b3-e46292c0f960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:49.140560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a69e15-4925-4959-a4b3-e46292c0f960 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:49.150321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 962/1893] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:49.155030 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957fd69-5165-47b1-83e6-e1dcc2b5217f admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:49.170411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957fd69-5165-47b1-83e6-e1dcc2b5217f admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:49.173557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a092d11-e8be-4733-b43e-36936fbc88d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:49.174509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a092d11-e8be-4733-b43e-36936fbc88d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:49.174802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a092d11-e8be-4733-b43e-36936fbc88d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:49.175272 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a092d11-e8be-4733-b43e-36936fbc88d1 admin admin] PATH_INFO: `/v3/projects/7370480bec7a431ea2577852f29c04e2/users/bf6d713139ed47fa90da6d13cf9073c3/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:49.192580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a092d11-e8be-4733-b43e-36936fbc88d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7370480bec7a431ea2577852f29c04e2, user_id=bf6d713139ed47fa90da6d13cf9073c3, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:49.195593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a092d11-e8be-4733-b43e-36936fbc88d1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:49.289974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 932/1894] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:49 2022] PUT /identity/v3/projects/7370480bec7a431ea2577852f29c04e2/users/bf6d713139ed47fa90da6d13cf9073c3/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:49.300963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7b6d9-a7ad-4261-a44a-f3674e945aa7 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:49.301396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7b6d9-a7ad-4261-a44a-f3674e945aa7 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:49.301682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7b6d9-a7ad-4261-a44a-f3674e945aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:49.331987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f8b7b6d9-a7ad-4261-a44a-f3674e945aa7 None None] MFA Rules not processed for user `bf6d713139ed47fa90da6d13cf9073c3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:49.333304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f8b7b6d9-a7ad-4261-a44a-f3674e945aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:49.406318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 963/1895] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:49.411057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a092d11-e8be-4733-b43e-36936fbc88d1 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:49.473695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a092d11-e8be-4733-b43e-36936fbc88d1 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:49.478305 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ed0c3-b4d2-459f-8276-7c4927112310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:49.478916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ed0c3-b4d2-459f-8276-7c4927112310 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:49.479198 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ed0c3-b4d2-459f-8276-7c4927112310 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:49.479473 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ed0c3-b4d2-459f-8276-7c4927112310 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:49.480142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ed0c3-b4d2-459f-8276-7c4927112310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:49.482050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ed0c3-b4d2-459f-8276-7c4927112310 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:49.622225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 933/1896] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:49 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:49.629757 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7b6d9-a7ad-4261-a44a-f3674e945aa7 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:49.637221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7b6d9-a7ad-4261-a44a-f3674e945aa7 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:49.693526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03617571-ab42-4cde-ab36-7bd9586247c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:49.694131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03617571-ab42-4cde-ab36-7bd9586247c4 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:49.694413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03617571-ab42-4cde-ab36-7bd9586247c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:49.694683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03617571-ab42-4cde-ab36-7bd9586247c4 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:49.695307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03617571-ab42-4cde-ab36-7bd9586247c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:49.697231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03617571-ab42-4cde-ab36-7bd9586247c4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:49.803749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 964/1897] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:49 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:49.809092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ed0c3-b4d2-459f-8276-7c4927112310 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:49.820555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ed0c3-b4d2-459f-8276-7c4927112310 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:49.825328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f82fe-41f1-443c-96a5-e423cc3bb105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:49.827335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f82fe-41f1-443c-96a5-e423cc3bb105 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:49.827335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f82fe-41f1-443c-96a5-e423cc3bb105 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:49.827532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:49.827639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f82fe-41f1-443c-96a5-e423cc3bb105 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:49.838813 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f82fe-41f1-443c-96a5-e423cc3bb105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:49.840542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f82fe-41f1-443c-96a5-e423cc3bb105 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:49.854929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 934/1898] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:49.858808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03617571-ab42-4cde-ab36-7bd9586247c4 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:49.869798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03617571-ab42-4cde-ab36-7bd9586247c4 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:49.875062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c26cf5-a062-471a-834e-70fc0efd2c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:49.878516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c26cf5-a062-471a-834e-70fc0efd2c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:49.878516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c26cf5-a062-471a-834e-70fc0efd2c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:49.878516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c26cf5-a062-471a-834e-70fc0efd2c00 admin admin] PATH_INFO: `/v3/projects/7f752284d3d7494ba8bc5308261c9ecb/users/990aad71ff574757aff907898b305f9a/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:49.897921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c26cf5-a062-471a-834e-70fc0efd2c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f752284d3d7494ba8bc5308261c9ecb, user_id=990aad71ff574757aff907898b305f9a, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:49.901812 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c26cf5-a062-471a-834e-70fc0efd2c00 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:50.015320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 965/1899] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:49 2022] PUT /identity/v3/projects/7f752284d3d7494ba8bc5308261c9ecb/users/990aad71ff574757aff907898b305f9a/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:50.019121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f82fe-41f1-443c-96a5-e423cc3bb105 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:50.033133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f82fe-41f1-443c-96a5-e423cc3bb105 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:50.106895 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dda9ea-518d-4b2d-ad46-6e454c30400b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:50.108028 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dda9ea-518d-4b2d-ad46-6e454c30400b admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:50.108540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dda9ea-518d-4b2d-ad46-6e454c30400b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:50.108894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dda9ea-518d-4b2d-ad46-6e454c30400b admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:50.109693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dda9ea-518d-4b2d-ad46-6e454c30400b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:50.111512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dda9ea-518d-4b2d-ad46-6e454c30400b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:50.125560 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 935/1900] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:50.129706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c26cf5-a062-471a-834e-70fc0efd2c00 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:50.140282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c26cf5-a062-471a-834e-70fc0efd2c00 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:50.144838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd238d-d51d-42cf-b7dd-dfc447eedc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:50.145477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd238d-d51d-42cf-b7dd-dfc447eedc57 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:50.145763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd238d-d51d-42cf-b7dd-dfc447eedc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:50.146038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd238d-d51d-42cf-b7dd-dfc447eedc57 admin admin] PATH_INFO: `/v3/projects/7f752284d3d7494ba8bc5308261c9ecb/users/990aad71ff574757aff907898b305f9a/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:50.148560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd238d-d51d-42cf-b7dd-dfc447eedc57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f752284d3d7494ba8bc5308261c9ecb, user_id=990aad71ff574757aff907898b305f9a, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:50.151161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd238d-d51d-42cf-b7dd-dfc447eedc57 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:50.161801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 966/1901] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:50 2022] PUT /identity/v3/projects/7f752284d3d7494ba8bc5308261c9ecb/users/990aad71ff574757aff907898b305f9a/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:50.172958 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb18a68-2047-4544-b87c-88ea22e42c1a None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:50.173299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb18a68-2047-4544-b87c-88ea22e42c1a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:50.173586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb18a68-2047-4544-b87c-88ea22e42c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:50.202678 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-edb18a68-2047-4544-b87c-88ea22e42c1a None None] MFA Rules not processed for user `990aad71ff574757aff907898b305f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:50.204026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-edb18a68-2047-4544-b87c-88ea22e42c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:50.284060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 936/1902] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:50.293384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd238d-d51d-42cf-b7dd-dfc447eedc57 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:50.295084 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6bcd238d-d51d-42cf-b7dd-dfc447eedc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:50.354005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd238d-d51d-42cf-b7dd-dfc447eedc57 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:50.357743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9446d-e75b-44bb-97ab-3d56d3e91cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:50.358297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9446d-e75b-44bb-97ab-3d56d3e91cbf service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:50.358583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9446d-e75b-44bb-97ab-3d56d3e91cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:50.358862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9446d-e75b-44bb-97ab-3d56d3e91cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:50.368100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9446d-e75b-44bb-97ab-3d56d3e91cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:50.369581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9446d-e75b-44bb-97ab-3d56d3e91cbf service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:50.379690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 967/1903] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:50.526877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb18a68-2047-4544-b87c-88ea22e42c1a None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:50.591241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb18a68-2047-4544-b87c-88ea22e42c1a None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:50.595853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:50.601750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:50.601750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:50.601750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] PATH_INFO: `/v3/users/bf6d713139ed47fa90da6d13cf9073c3` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:50.601750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:50.601750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf6d713139ed47fa90da6d13cf9073c3)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:50.602216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:50.826638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf6d713139ed47fa90da6d13cf9073c3'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:50.827302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf6d713139ed47fa90da6d13cf9073c3'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:50.839337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf6d713139ed47fa90da6d13cf9073c3'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:50.932494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf6d713139ed47fa90da6d13cf9073c3'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:50.941529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf6d713139ed47fa90da6d13cf9073c3'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:50.947505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 937/1904] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:50 2022] DELETE /identity/v3/users/bf6d713139ed47fa90da6d13cf9073c3 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:50.952053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245a5dd-5b6b-43f5-9fee-7c3cdb5d6a20 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:50.952528 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245a5dd-5b6b-43f5-9fee-7c3cdb5d6a20 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:50.952644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245a5dd-5b6b-43f5-9fee-7c3cdb5d6a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:50.971934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-4245a5dd-5b6b-43f5-9fee-7c3cdb5d6a20 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:50.973153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4245a5dd-5b6b-43f5-9fee-7c3cdb5d6a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:51.031844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 968/1905] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:51.045443 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:51.047087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:51.111024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b0d82-9f67-4542-95fe-f04c722770fb admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:51.119506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cc8ff-8064-497a-98cb-03fe4313e4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:51.119677 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cc8ff-8064-497a-98cb-03fe4313e4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:51.119858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cc8ff-8064-497a-98cb-03fe4313e4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:51.123421 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cc8ff-8064-497a-98cb-03fe4313e4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:51.148521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048cc8ff-8064-497a-98cb-03fe4313e4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:51.150140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048cc8ff-8064-497a-98cb-03fe4313e4e5 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:51.163719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 938/1906] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:51.677516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245a5dd-5b6b-43f5-9fee-7c3cdb5d6a20 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:51.679093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4245a5dd-5b6b-43f5-9fee-7c3cdb5d6a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:51.693251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245a5dd-5b6b-43f5-9fee-7c3cdb5d6a20 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:51.698776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:51.699506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:51.699935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:51.700299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] PATH_INFO: `/v3/projects/7370480bec7a431ea2577852f29c04e2` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:51.701619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7370480bec7a431ea2577852f29c04e2)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:51.703564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:51.712908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:51.819960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7370480bec7a431ea2577852f29c04e2', 'request_id': 'req-833dfc23-27ae-436d-ab94-a859df0c78fe'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:51.914959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7370480bec7a431ea2577852f29c04e2', 'request_id': 'req-833dfc23-27ae-436d-ab94-a859df0c78fe'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:51.922871 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7370480bec7a431ea2577852f29c04e2', 'request_id': 'req-833dfc23-27ae-436d-ab94-a859df0c78fe'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:51.934675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] The token cache is being invalidate because project 7370480bec7a431ea2577852f29c04e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:51.934951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:51.935619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:51.937131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 969/1907] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:51 2022] DELETE /identity/v3/projects/7370480bec7a431ea2577852f29c04e2 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:51.941336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cc8ff-8064-497a-98cb-03fe4313e4e5 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:51.951154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cc8ff-8064-497a-98cb-03fe4313e4e5 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:52.009266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269997fe-d1f1-4bcf-b46e-2aa0ba00ae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:52.009904 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269997fe-d1f1-4bcf-b46e-2aa0ba00ae31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:52.010192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269997fe-d1f1-4bcf-b46e-2aa0ba00ae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:52.010474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269997fe-d1f1-4bcf-b46e-2aa0ba00ae31 admin admin] PATH_INFO: `/v3/users/990aad71ff574757aff907898b305f9a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:52.011682 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269997fe-d1f1-4bcf-b46e-2aa0ba00ae31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=990aad71ff574757aff907898b305f9a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:52.013747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269997fe-d1f1-4bcf-b46e-2aa0ba00ae31 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:52.240348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-269997fe-d1f1-4bcf-b46e-2aa0ba00ae31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '990aad71ff574757aff907898b305f9a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:52.241040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-269997fe-d1f1-4bcf-b46e-2aa0ba00ae31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '990aad71ff574757aff907898b305f9a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:52.252957 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-269997fe-d1f1-4bcf-b46e-2aa0ba00ae31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '990aad71ff574757aff907898b305f9a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:52.386875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-269997fe-d1f1-4bcf-b46e-2aa0ba00ae31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '990aad71ff574757aff907898b305f9a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:52.398883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-269997fe-d1f1-4bcf-b46e-2aa0ba00ae31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '990aad71ff574757aff907898b305f9a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:52.409247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 939/1908] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:50:51 2022] DELETE /identity/v3/users/990aad71ff574757aff907898b305f9a => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:52.687372 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:52.831919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:52.833427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:52.847456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dfc23-27ae-436d-ab94-a859df0c78fe admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:52.921954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:52.922598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:52.922886 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:52.923165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] PATH_INFO: `/v3/projects/7f752284d3d7494ba8bc5308261c9ecb` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:52.924337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f752284d3d7494ba8bc5308261c9ecb)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:52.926311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:53.053182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f752284d3d7494ba8bc5308261c9ecb', 'request_id': 'req-435fbb68-8935-48c7-86d9-aacd7077993b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:53.146293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f752284d3d7494ba8bc5308261c9ecb', 'request_id': 'req-435fbb68-8935-48c7-86d9-aacd7077993b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:53.153462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f752284d3d7494ba8bc5308261c9ecb', 'request_id': 'req-435fbb68-8935-48c7-86d9-aacd7077993b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:53.164906 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] The token cache is being invalidate because project 7f752284d3d7494ba8bc5308261c9ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:50:53.165131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:53.165800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-435fbb68-8935-48c7-86d9-aacd7077993b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:50:53.167323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 970/1909] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:50:52 2022] DELETE /identity/v3/projects/7f752284d3d7494ba8bc5308261c9ecb => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:50:53.173185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18cfd4-1be0-4e1b-8616-bd35129ed5b8 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:53.173694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18cfd4-1be0-4e1b-8616-bd35129ed5b8 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:53.173795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18cfd4-1be0-4e1b-8616-bd35129ed5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:53.193231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-eb18cfd4-1be0-4e1b-8616-bd35129ed5b8 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:53.194464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-eb18cfd4-1be0-4e1b-8616-bd35129ed5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:53.253845 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 940/1910] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:53.261898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f870a3-a42e-46d7-943b-edaf327ccb53 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:53.262194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f870a3-a42e-46d7-943b-edaf327ccb53 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:53.262508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f870a3-a42e-46d7-943b-edaf327ccb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:53.283757 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-59f870a3-a42e-46d7-943b-edaf327ccb53 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:53.285100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-59f870a3-a42e-46d7-943b-edaf327ccb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:53.293088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 971/1911] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:53.298623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18cfd4-1be0-4e1b-8616-bd35129ed5b8 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:53.308496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18cfd4-1be0-4e1b-8616-bd35129ed5b8 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:53.313965 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddb9be-b500-48dd-a9fd-7d5bc83158e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:53.314568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddb9be-b500-48dd-a9fd-7d5bc83158e5 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:53.314858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddb9be-b500-48dd-a9fd-7d5bc83158e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:53.315142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddb9be-b500-48dd-a9fd-7d5bc83158e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:53.315796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddb9be-b500-48dd-a9fd-7d5bc83158e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:50:53.316153 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddb9be-b500-48dd-a9fd-7d5bc83158e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:53.317793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddb9be-b500-48dd-a9fd-7d5bc83158e5 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:53.330243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 941/1912] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:50:53.334709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f870a3-a42e-46d7-943b-edaf327ccb53 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:53.344300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f870a3-a42e-46d7-943b-edaf327ccb53 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:53.349066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e8d6c-09fd-43f8-aa5c-0e1b63c29fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:53.349663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e8d6c-09fd-43f8-aa5c-0e1b63c29fbf admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:53.349949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e8d6c-09fd-43f8-aa5c-0e1b63c29fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:53.350228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e8d6c-09fd-43f8-aa5c-0e1b63c29fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:53.350856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e8d6c-09fd-43f8-aa5c-0e1b63c29fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:53.352927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e8d6c-09fd-43f8-aa5c-0e1b63c29fbf admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:53.535068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 972/1913] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:50:53 2022] POST /identity/v3/projects => generated 375 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:53.539574 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddb9be-b500-48dd-a9fd-7d5bc83158e5 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:53.552491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddb9be-b500-48dd-a9fd-7d5bc83158e5 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:53.716431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05f1d6-ba00-4156-8eb5-87d72362a8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:53.716431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f1d6-ba00-4156-8eb5-87d72362a8a6 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:53.716431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f1d6-ba00-4156-8eb5-87d72362a8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:53.716431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f1d6-ba00-4156-8eb5-87d72362a8a6 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:53.717094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05f1d6-ba00-4156-8eb5-87d72362a8a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:53.717094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05f1d6-ba00-4156-8eb5-87d72362a8a6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:53.796078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 942/1914] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:50:53 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:50:53.800323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e8d6c-09fd-43f8-aa5c-0e1b63c29fbf admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:53.811910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e8d6c-09fd-43f8-aa5c-0e1b63c29fbf admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:53.816792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e3fc-e483-4e04-8179-da14e5a2ec02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:53.822329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e3fc-e483-4e04-8179-da14e5a2ec02 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:53.822329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e3fc-e483-4e04-8179-da14e5a2ec02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:53.822329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e3fc-e483-4e04-8179-da14e5a2ec02 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:53.822329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15e3fc-e483-4e04-8179-da14e5a2ec02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:53.822329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15e3fc-e483-4e04-8179-da14e5a2ec02 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:53.831764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 973/1915] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:50:53.836084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05f1d6-ba00-4156-8eb5-87d72362a8a6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:53.846543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05f1d6-ba00-4156-8eb5-87d72362a8a6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:53.851571 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a3b32-7cf5-4db8-aa85-08207a918f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:53.852250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3b32-7cf5-4db8-aa85-08207a918f49 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:53.852528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3b32-7cf5-4db8-aa85-08207a918f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:53.853251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3b32-7cf5-4db8-aa85-08207a918f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:53.853486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a3b32-7cf5-4db8-aa85-08207a918f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:53.855036 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a3b32-7cf5-4db8-aa85-08207a918f49 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:53.869878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e3fc-e483-4e04-8179-da14e5a2ec02 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:53.870472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 943/1916] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:50:53.881175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e3fc-e483-4e04-8179-da14e5a2ec02 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:53.892543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d35a4c-6c61-4904-877f-57962afd0c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:53.893151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d35a4c-6c61-4904-877f-57962afd0c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:53.893478 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d35a4c-6c61-4904-877f-57962afd0c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:53.893779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d35a4c-6c61-4904-877f-57962afd0c6e admin admin] PATH_INFO: `/v3/projects/8c136b19c69c47388515601c08251a27/users/4996594a6fe04173bbefb2b92e8ccb58/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:53.910645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d35a4c-6c61-4904-877f-57962afd0c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c136b19c69c47388515601c08251a27, user_id=4996594a6fe04173bbefb2b92e8ccb58, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:53.914128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d35a4c-6c61-4904-877f-57962afd0c6e admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:53.993391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d21e4-81ea-4c31-a1b1-6cedf4f69da0 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:53.995843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 974/1917] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:50:53 2022] PUT /identity/v3/projects/8c136b19c69c47388515601c08251a27/users/4996594a6fe04173bbefb2b92e8ccb58/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:50:54.004118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d21e4-81ea-4c31-a1b1-6cedf4f69da0 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:54.004517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d21e4-81ea-4c31-a1b1-6cedf4f69da0 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:54.005695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55ac37-9cb1-4223-aa2d-dc8483ce5e38 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:54.006081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55ac37-9cb1-4223-aa2d-dc8483ce5e38 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:54.006385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 944/1918] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 15:50:53 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:50:54.006919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55ac37-9cb1-4223-aa2d-dc8483ce5e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:54.018428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69f986-4877-4197-b511-a14a697c1dc1 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:54.018815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69f986-4877-4197-b511-a14a697c1dc1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:54.019031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69f986-4877-4197-b511-a14a697c1dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:54.047473 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ee69f986-4877-4197-b511-a14a697c1dc1 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:54.047473 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ee69f986-4877-4197-b511-a14a697c1dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:54.055966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3d55ac37-9cb1-4223-aa2d-dc8483ce5e38 None None] MFA Rules not processed for user `4996594a6fe04173bbefb2b92e8ccb58`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:54.062849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3d55ac37-9cb1-4223-aa2d-dc8483ce5e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:54.171290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 945/1919] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 15:50:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:54.195290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=78,keystone:INSERT=5,keystone:DELETE=17 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:54.199575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69f986-4877-4197-b511-a14a697c1dc1 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:54.201076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ee69f986-4877-4197-b511-a14a697c1dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:54.214115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 975/1920] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:50:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:54.215304 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69f986-4877-4197-b511-a14a697c1dc1 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:54.237457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55ac37-9cb1-4223-aa2d-dc8483ce5e38 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:54.237745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1e6a2-9d78-4613-8719-ce1ec3062e16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:54.238421 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1e6a2-9d78-4613-8719-ce1ec3062e16 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:54.239077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1e6a2-9d78-4613-8719-ce1ec3062e16 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:54.239365 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1e6a2-9d78-4613-8719-ce1ec3062e16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:54.241495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3d55ac37-9cb1-4223-aa2d-dc8483ce5e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:54.250857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1e6a2-9d78-4613-8719-ce1ec3062e16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:54.253208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1e6a2-9d78-4613-8719-ce1ec3062e16 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:54.266044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 946/1921] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:50:54.307315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:54.336038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55ac37-9cb1-4223-aa2d-dc8483ce5e38 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:54.340460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba045c-5f88-47e6-beb4-8e638a2b7697 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cef193a54eff44f2b53716e3ec1a29cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:54.340883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba045c-5f88-47e6-beb4-8e638a2b7697 service glance] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:54.340883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba045c-5f88-47e6-beb4-8e638a2b7697 service glance] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:54.341121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba045c-5f88-47e6-beb4-8e638a2b7697 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:54.359619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba045c-5f88-47e6-beb4-8e638a2b7697 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:54.361688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba045c-5f88-47e6-beb4-8e638a2b7697 service glance] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:54.375788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 976/1922] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:57.796173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:58.016071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:58.832625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:59.358170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:59.461150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:00.575260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:01.473277 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:02.363973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:02.427681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1e6a2-9d78-4613-8719-ce1ec3062e16 service nova] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:02.429340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-19d1e6a2-9d78-4613-8719-ce1ec3062e16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:02.524414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1e6a2-9d78-4613-8719-ce1ec3062e16 service nova] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:02.530134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:02.531171 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:02.535793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:02.535793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] PATH_INFO: `/v3/users/78f3db4069f3434ea1ab4553ab6de815` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:02.535793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78f3db4069f3434ea1ab4553ab6de815)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:02.536250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:02.768376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78f3db4069f3434ea1ab4553ab6de815'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:02.769053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78f3db4069f3434ea1ab4553ab6de815'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:02.781650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78f3db4069f3434ea1ab4553ab6de815'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:02.880827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78f3db4069f3434ea1ab4553ab6de815'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:02.904970 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78f3db4069f3434ea1ab4553ab6de815'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:02.904970 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 947/1923] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:02 2022] DELETE /identity/v3/users/78f3db4069f3434ea1ab4553ab6de815 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:02.908103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd379290-3c01-45af-a61c-434f5d749360 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:02.908531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd379290-3c01-45af-a61c-434f5d749360 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:02.908791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd379290-3c01-45af-a61c-434f5d749360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:02.931738 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-cd379290-3c01-45af-a61c-434f5d749360 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:02.931893 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cd379290-3c01-45af-a61c-434f5d749360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:03.009616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 977/1924] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:03.019815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:03.021337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:03.081962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a742f53-6e26-430d-9e04-6bf5093ff706 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:03.085849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e5d8e-0a26-4ae1-8c78-ce479a9b5f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:03.086469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e5d8e-0a26-4ae1-8c78-ce479a9b5f05 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:03.086755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e5d8e-0a26-4ae1-8c78-ce479a9b5f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:03.087061 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e5d8e-0a26-4ae1-8c78-ce479a9b5f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:03.099805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e5d8e-0a26-4ae1-8c78-ce479a9b5f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:03.101326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e5d8e-0a26-4ae1-8c78-ce479a9b5f05 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:03.111706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 948/1925] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:03.375824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd379290-3c01-45af-a61c-434f5d749360 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:03.377358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cd379290-3c01-45af-a61c-434f5d749360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:03.395907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd379290-3c01-45af-a61c-434f5d749360 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:03.401474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfde4b-c17d-4d67-9b2f-94bf82d55c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:03.402272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfde4b-c17d-4d67-9b2f-94bf82d55c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:03.402876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfde4b-c17d-4d67-9b2f-94bf82d55c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:03.403156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfde4b-c17d-4d67-9b2f-94bf82d55c4a admin admin] PATH_INFO: `/v3/users/f4857491584144d6bebc96706fbc7128` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:03.404405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfde4b-c17d-4d67-9b2f-94bf82d55c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4857491584144d6bebc96706fbc7128)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:03.407025 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfde4b-c17d-4d67-9b2f-94bf82d55c4a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:03.655966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e5d8e-0a26-4ae1-8c78-ce479a9b5f05 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:03.656987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-582e5d8e-0a26-4ae1-8c78-ce479a9b5f05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:03.685296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e5d8e-0a26-4ae1-8c78-ce479a9b5f05 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:03.685786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:03.690581 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:03.691678 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:03.692010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:03.692400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] PATH_INFO: `/v3/projects/e371cf1572db4921b33fb6f4d586d660` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:03.693603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e371cf1572db4921b33fb6f4d586d660)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:03.696737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:03.714594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2ddfde4b-c17d-4d67-9b2f-94bf82d55c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4857491584144d6bebc96706fbc7128'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:03.729150 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2ddfde4b-c17d-4d67-9b2f-94bf82d55c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4857491584144d6bebc96706fbc7128'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:03.730390 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2ddfde4b-c17d-4d67-9b2f-94bf82d55c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4857491584144d6bebc96706fbc7128'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:03.731111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2ddfde4b-c17d-4d67-9b2f-94bf82d55c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4857491584144d6bebc96706fbc7128'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:03.807412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2ddfde4b-c17d-4d67-9b2f-94bf82d55c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4857491584144d6bebc96706fbc7128'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:03.822520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 978/1926] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:03 2022] DELETE /identity/v3/users/f4857491584144d6bebc96706fbc7128 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:03.828196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c1ab0-a22a-4dba-a020-8d6d1135d154 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:03.828431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c1ab0-a22a-4dba-a020-8d6d1135d154 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:03.828719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c1ab0-a22a-4dba-a020-8d6d1135d154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:03.833707 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e371cf1572db4921b33fb6f4d586d660', 'request_id': 'req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:03.840768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e371cf1572db4921b33fb6f4d586d660', 'request_id': 'req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:03.848464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e371cf1572db4921b33fb6f4d586d660', 'request_id': 'req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:03.856876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ac4c1ab0-a22a-4dba-a020-8d6d1135d154 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:03.858525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ac4c1ab0-a22a-4dba-a020-8d6d1135d154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:03.908788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:03.920221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 979/1927] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:03.931777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c1ab0-a22a-4dba-a020-8d6d1135d154 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:03.937005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ac4c1ab0-a22a-4dba-a020-8d6d1135d154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:03.971150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] The token cache is being invalidate because project e371cf1572db4921b33fb6f4d586d660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:03.971442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:03.972291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:03.974837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 949/1928] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:03 2022] DELETE /identity/v3/projects/e371cf1572db4921b33fb6f4d586d660 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:03.979110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:03.983787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:04.007371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c1ab0-a22a-4dba-a020-8d6d1135d154 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:04.016845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13250b-0e85-4d3b-aba7-4372eb08f90c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:04.019230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13250b-0e85-4d3b-aba7-4372eb08f90c service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:04.019230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13250b-0e85-4d3b-aba7-4372eb08f90c service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:04.019230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13250b-0e85-4d3b-aba7-4372eb08f90c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:04.020326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5a13250b-0e85-4d3b-aba7-4372eb08f90c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:04.033402 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13250b-0e85-4d3b-aba7-4372eb08f90c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:04.035331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13250b-0e85-4d3b-aba7-4372eb08f90c service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:04.045920 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 980/1929] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:51:04.058151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde4ffb-3b6e-443a-9e1d-5460f1befbc1 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:04.064067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:04.064786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:04.065177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:04.065548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] PATH_INFO: `/v3/users/2f5d561a54224953844be84c371f5cd4` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:04.066875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f5d561a54224953844be84c371f5cd4)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:04.068991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:04.254615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f5d561a54224953844be84c371f5cd4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:04.255309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f5d561a54224953844be84c371f5cd4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:04.269465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f5d561a54224953844be84c371f5cd4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:04.375811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f5d561a54224953844be84c371f5cd4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:04.389121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f5d561a54224953844be84c371f5cd4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:04.401735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 950/1930] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:03 2022] DELETE /identity/v3/users/2f5d561a54224953844be84c371f5cd4 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:04.603129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13250b-0e85-4d3b-aba7-4372eb08f90c service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:04.604822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5a13250b-0e85-4d3b-aba7-4372eb08f90c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:04.618694 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13250b-0e85-4d3b-aba7-4372eb08f90c service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:04.687759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:04.688458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:04.688743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:04.689019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] PATH_INFO: `/v3/projects/d50834ed551c4b589815a44354bd2870` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:04.690168 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d50834ed551c4b589815a44354bd2870)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:04.692146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:04.892147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd50834ed551c4b589815a44354bd2870', 'request_id': 'req-f308806c-5362-4384-bc73-561ab120b9a5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:04.995425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd50834ed551c4b589815a44354bd2870', 'request_id': 'req-f308806c-5362-4384-bc73-561ab120b9a5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.003002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd50834ed551c4b589815a44354bd2870', 'request_id': 'req-f308806c-5362-4384-bc73-561ab120b9a5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.013772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] The token cache is being invalidate because project d50834ed551c4b589815a44354bd2870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:05.014081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.014752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.016444 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 981/1931] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:04 2022] DELETE /identity/v3/projects/d50834ed551c4b589815a44354bd2870 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:05.021111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:05.023419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:05.045506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613b382-48ea-4f38-b785-fad7b33a6233 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:05.088617 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:05.090888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:05.108292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:05.108920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:05.109810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:05.110098 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] PATH_INFO: `/v3/users/fa5dbc82446d485e93e56b9eff6e0e55` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:05.111300 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5dbc82446d485e93e56b9eff6e0e55)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:05.114101 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:05.167890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308806c-5362-4384-bc73-561ab120b9a5 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:05.174222 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:05.174883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:05.175170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:05.175453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] PATH_INFO: `/v3/projects/aa0e6435408540d38ba71d5a436ee25e` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:05.176930 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa0e6435408540d38ba71d5a436ee25e)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:05.178858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:05.322393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5dbc82446d485e93e56b9eff6e0e55'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.323296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5dbc82446d485e93e56b9eff6e0e55'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.323682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa0e6435408540d38ba71d5a436ee25e', 'request_id': 'req-135eb870-f151-497b-b661-eaa2d6e8b75d'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.335820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5dbc82446d485e93e56b9eff6e0e55'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.409804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa0e6435408540d38ba71d5a436ee25e', 'request_id': 'req-135eb870-f151-497b-b661-eaa2d6e8b75d'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.416568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa0e6435408540d38ba71d5a436ee25e', 'request_id': 'req-135eb870-f151-497b-b661-eaa2d6e8b75d'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.427642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] The token cache is being invalidate because project aa0e6435408540d38ba71d5a436ee25e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:05.427969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.428656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-135eb870-f151-497b-b661-eaa2d6e8b75d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.430165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 982/1932] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:05 2022] DELETE /identity/v3/projects/aa0e6435408540d38ba71d5a436ee25e => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:05.435064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5889d-b615-4612-8d0a-3fb661f36ca8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:05.435385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5889d-b615-4612-8d0a-3fb661f36ca8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:05.435671 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5889d-b615-4612-8d0a-3fb661f36ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:05.454851 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-05b5889d-b615-4612-8d0a-3fb661f36ca8 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:05.456092 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-05b5889d-b615-4612-8d0a-3fb661f36ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:05.509320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5dbc82446d485e93e56b9eff6e0e55'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.518296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5dbc82446d485e93e56b9eff6e0e55'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:05.520658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 983/1933] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:05.525397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 951/1934] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:05 2022] DELETE /identity/v3/users/fa5dbc82446d485e93e56b9eff6e0e55 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:05.530436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e133ff-206f-4311-b171-6d3d1272788d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:05.530715 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e133ff-206f-4311-b171-6d3d1272788d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:05.531004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e133ff-206f-4311-b171-6d3d1272788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:05.554108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-62e133ff-206f-4311-b171-6d3d1272788d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:05.555345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-62e133ff-206f-4311-b171-6d3d1272788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:05.626984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 984/1935] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:05.630930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:05.642400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c8e7c-3aec-4006-b5ca-60b93418b4d6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:05.652467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50d0a2-1ac0-4222-bf7f-ca02d64ad612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:05.652467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50d0a2-1ac0-4222-bf7f-ca02d64ad612 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:05.652467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50d0a2-1ac0-4222-bf7f-ca02d64ad612 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:05.652467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50d0a2-1ac0-4222-bf7f-ca02d64ad612 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:05.652821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50d0a2-1ac0-4222-bf7f-ca02d64ad612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:05.652821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50d0a2-1ac0-4222-bf7f-ca02d64ad612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:05.652821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50d0a2-1ac0-4222-bf7f-ca02d64ad612 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:05.669034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 952/1936] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:51:05.670403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e133ff-206f-4311-b171-6d3d1272788d None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:05.680076 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e133ff-206f-4311-b171-6d3d1272788d None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:05.685349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a80b3c-0441-4b92-9257-a5e30a229856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:05.685986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a80b3c-0441-4b92-9257-a5e30a229856 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:05.686269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a80b3c-0441-4b92-9257-a5e30a229856 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:05.686551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a80b3c-0441-4b92-9257-a5e30a229856 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:05.687188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a80b3c-0441-4b92-9257-a5e30a229856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:05.689198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a80b3c-0441-4b92-9257-a5e30a229856 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:05.821641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:05.832071 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 985/1937] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:05 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:05.833316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50d0a2-1ac0-4222-bf7f-ca02d64ad612 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:05.845189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50d0a2-1ac0-4222-bf7f-ca02d64ad612 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:05.905167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf3155-caf8-43cf-8c98-261d3819ed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:05.905798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf3155-caf8-43cf-8c98-261d3819ed61 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:05.906085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf3155-caf8-43cf-8c98-261d3819ed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:05.906367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf3155-caf8-43cf-8c98-261d3819ed61 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:05.907023 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf3155-caf8-43cf-8c98-261d3819ed61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:05.909014 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf3155-caf8-43cf-8c98-261d3819ed61 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:06.031239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 953/1938] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:05 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:51:06.036044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a80b3c-0441-4b92-9257-a5e30a229856 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:06.046452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a80b3c-0441-4b92-9257-a5e30a229856 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:06.051443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b9971-9322-437f-ade0-463120735d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:06.052089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b9971-9322-437f-ade0-463120735d82 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:06.052375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b9971-9322-437f-ade0-463120735d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:06.052655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b9971-9322-437f-ade0-463120735d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:06.053312 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b9971-9322-437f-ade0-463120735d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:06.054931 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b9971-9322-437f-ade0-463120735d82 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:06.064700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf3155-caf8-43cf-8c98-261d3819ed61 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:06.067315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-debf3155-caf8-43cf-8c98-261d3819ed61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:06.071979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 986/1939] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:06.073096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b9971-9322-437f-ade0-463120735d82 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:06.084598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b9971-9322-437f-ade0-463120735d82 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:06.091052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4328f2e-1305-4bf1-a5f6-97b772967f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:06.091713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4328f2e-1305-4bf1-a5f6-97b772967f88 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:06.092039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4328f2e-1305-4bf1-a5f6-97b772967f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:06.092320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4328f2e-1305-4bf1-a5f6-97b772967f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:06.092973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4328f2e-1305-4bf1-a5f6-97b772967f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:06.094780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4328f2e-1305-4bf1-a5f6-97b772967f88 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:06.096388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf3155-caf8-43cf-8c98-261d3819ed61 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:06.105020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:06.105020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:06.105020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:06.105020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] PATH_INFO: `/v3/projects/18b3c9ce84314cdf858155362075f336` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:06.105523 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b3c9ce84314cdf858155362075f336)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:06.106735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:06.107518 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 987/1940] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:51:06.111591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4328f2e-1305-4bf1-a5f6-97b772967f88 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:06.121823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4328f2e-1305-4bf1-a5f6-97b772967f88 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:06.127342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ac1c3-a4cd-4419-a4b3-4672c1308267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:06.128255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ac1c3-a4cd-4419-a4b3-4672c1308267 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:06.128549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ac1c3-a4cd-4419-a4b3-4672c1308267 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:06.128831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ac1c3-a4cd-4419-a4b3-4672c1308267 admin admin] PATH_INFO: `/v3/projects/6120fe64de3946849ac8204aa6828074/users/a430ea6c831b4f5ab6a913ae240ae88d/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:06.144980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ac1c3-a4cd-4419-a4b3-4672c1308267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6120fe64de3946849ac8204aa6828074, user_id=a430ea6c831b4f5ab6a913ae240ae88d, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:06.147591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ac1c3-a4cd-4419-a4b3-4672c1308267 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:06.264852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 988/1941] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:06 2022] PUT /identity/v3/projects/6120fe64de3946849ac8204aa6828074/users/a430ea6c831b4f5ab6a913ae240ae88d/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:06.275276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460166-3c07-4fd7-b454-43a6f8029271 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:06.275568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460166-3c07-4fd7-b454-43a6f8029271 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:06.275851 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460166-3c07-4fd7-b454-43a6f8029271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:06.281158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b3c9ce84314cdf858155362075f336', 'request_id': 'req-bdba11ce-a90f-41f9-b520-5691103ed608'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:06.291230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b3c9ce84314cdf858155362075f336', 'request_id': 'req-bdba11ce-a90f-41f9-b520-5691103ed608'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:06.300406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b3c9ce84314cdf858155362075f336', 'request_id': 'req-bdba11ce-a90f-41f9-b520-5691103ed608'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:06.316800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-7f460166-3c07-4fd7-b454-43a6f8029271 None None] MFA Rules not processed for user `a430ea6c831b4f5ab6a913ae240ae88d`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:06.318021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-7f460166-3c07-4fd7-b454-43a6f8029271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:06.401445 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 989/1942] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:06.413508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f460166-3c07-4fd7-b454-43a6f8029271 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:06.415332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-7f460166-3c07-4fd7-b454-43a6f8029271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:06.416077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] The token cache is being invalidate because project 18b3c9ce84314cdf858155362075f336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:06.417433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:06.418369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bdba11ce-a90f-41f9-b520-5691103ed608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:06.421696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 954/1943] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:06 2022] DELETE /identity/v3/projects/18b3c9ce84314cdf858155362075f336 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:06.427212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1871f-a293-4a29-ac2c-e18c44f1ef45 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:06.427758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1871f-a293-4a29-ac2c-e18c44f1ef45 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:06.428128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1871f-a293-4a29-ac2c-e18c44f1ef45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:06.456864 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-3fc1871f-a293-4a29-ac2c-e18c44f1ef45 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:06.458217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3fc1871f-a293-4a29-ac2c-e18c44f1ef45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:06.501944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:06.510169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f460166-3c07-4fd7-b454-43a6f8029271 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:06.515017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed38ae-cd1a-43c2-82da-052c89678d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:06.515717 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed38ae-cd1a-43c2-82da-052c89678d83 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:06.516063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed38ae-cd1a-43c2-82da-052c89678d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:06.516536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed38ae-cd1a-43c2-82da-052c89678d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:06.518409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a9ed38ae-cd1a-43c2-82da-052c89678d83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:06.550142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed38ae-cd1a-43c2-82da-052c89678d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:06.551654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed38ae-cd1a-43c2-82da-052c89678d83 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:06.556773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 955/1944] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:06.563051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 990/1945] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:06.566400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391e21-c3b5-4d96-beb2-e5b36689e93e None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:06.567115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391e21-c3b5-4d96-beb2-e5b36689e93e None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:06.567769 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391e21-c3b5-4d96-beb2-e5b36689e93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:06.606422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-83391e21-c3b5-4d96-beb2-e5b36689e93e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:06.608749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-83391e21-c3b5-4d96-beb2-e5b36689e93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:06.618194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 956/1946] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:06.624909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed38ae-cd1a-43c2-82da-052c89678d83 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:06.650377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed38ae-cd1a-43c2-82da-052c89678d83 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:06.656755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c90b07-b145-4bcf-86bd-faebd2976886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:06.657484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c90b07-b145-4bcf-86bd-faebd2976886 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:06.657878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c90b07-b145-4bcf-86bd-faebd2976886 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:06.658234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c90b07-b145-4bcf-86bd-faebd2976886 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:06.659148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c90b07-b145-4bcf-86bd-faebd2976886 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:06.659526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c90b07-b145-4bcf-86bd-faebd2976886 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:06.661311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c90b07-b145-4bcf-86bd-faebd2976886 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:06.674448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 991/1947] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:51:06.679065 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83391e21-c3b5-4d96-beb2-e5b36689e93e None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:06.689367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83391e21-c3b5-4d96-beb2-e5b36689e93e None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:06.694170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2a804-f13b-4d06-8632-4736bd91c0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:06.694791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2a804-f13b-4d06-8632-4736bd91c0b8 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:06.695109 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2a804-f13b-4d06-8632-4736bd91c0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:06.695392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2a804-f13b-4d06-8632-4736bd91c0b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:06.696103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2a804-f13b-4d06-8632-4736bd91c0b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:06.698068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2a804-f13b-4d06-8632-4736bd91c0b8 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:06.843921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 957/1948] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:06 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:06.860123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c90b07-b145-4bcf-86bd-faebd2976886 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:06.873870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c90b07-b145-4bcf-86bd-faebd2976886 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:06.962963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d17ee-f284-47ce-9268-358ad7363520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:06.963904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d17ee-f284-47ce-9268-358ad7363520 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:06.964203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d17ee-f284-47ce-9268-358ad7363520 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:06.964495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d17ee-f284-47ce-9268-358ad7363520 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:06.965144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d17ee-f284-47ce-9268-358ad7363520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:06.967122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d17ee-f284-47ce-9268-358ad7363520 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:07.157201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 992/1949] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:06 2022] POST /identity/v3/users => generated 294 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:07.161674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2a804-f13b-4d06-8632-4736bd91c0b8 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:07.172490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2a804-f13b-4d06-8632-4736bd91c0b8 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:07.177609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42517b-c7f7-4c8c-abfd-ecf9d94539b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:07.178460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42517b-c7f7-4c8c-abfd-ecf9d94539b4 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:07.179028 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42517b-c7f7-4c8c-abfd-ecf9d94539b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:07.179474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42517b-c7f7-4c8c-abfd-ecf9d94539b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:07.180357 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42517b-c7f7-4c8c-abfd-ecf9d94539b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:07.187270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42517b-c7f7-4c8c-abfd-ecf9d94539b4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:07.196148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 958/1950] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:07.197443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d17ee-f284-47ce-9268-358ad7363520 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:07.209964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d17ee-f284-47ce-9268-358ad7363520 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:07.219320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1ecd2-173b-4ef5-93f9-59f6dcc8552b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:07.219984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1ecd2-173b-4ef5-93f9-59f6dcc8552b admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:07.220274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1ecd2-173b-4ef5-93f9-59f6dcc8552b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:07.220556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1ecd2-173b-4ef5-93f9-59f6dcc8552b admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:07.221252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1ecd2-173b-4ef5-93f9-59f6dcc8552b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:07.222867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1ecd2-173b-4ef5-93f9-59f6dcc8552b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:07.234376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 993/1951] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:07.238924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42517b-c7f7-4c8c-abfd-ecf9d94539b4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:07.249646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42517b-c7f7-4c8c-abfd-ecf9d94539b4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:07.254518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7348f-9699-4f4b-9fc7-72f924ea3135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:07.255281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7348f-9699-4f4b-9fc7-72f924ea3135 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:07.255654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7348f-9699-4f4b-9fc7-72f924ea3135 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:07.255982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7348f-9699-4f4b-9fc7-72f924ea3135 admin admin] PATH_INFO: `/v3/projects/c3ae52c6349b4cf2990b7acaf7b31266/users/31ef2c36acad4e1a8e5b96c7ce0f1edd/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:07.271335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7348f-9699-4f4b-9fc7-72f924ea3135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3ae52c6349b4cf2990b7acaf7b31266, user_id=31ef2c36acad4e1a8e5b96c7ce0f1edd, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:07.278539 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7348f-9699-4f4b-9fc7-72f924ea3135 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:07.399471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 959/1952] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:07 2022] PUT /identity/v3/projects/c3ae52c6349b4cf2990b7acaf7b31266/users/31ef2c36acad4e1a8e5b96c7ce0f1edd/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:07.408176 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49042cd9-52d0-459a-b793-2ff7e5293d80 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:07.408467 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49042cd9-52d0-459a-b793-2ff7e5293d80 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:07.408747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49042cd9-52d0-459a-b793-2ff7e5293d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:07.449473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-49042cd9-52d0-459a-b793-2ff7e5293d80 None None] MFA Rules not processed for user `31ef2c36acad4e1a8e5b96c7ce0f1edd`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:07.450828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-49042cd9-52d0-459a-b793-2ff7e5293d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:07.568768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 994/1953] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:07.580328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7348f-9699-4f4b-9fc7-72f924ea3135 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:07.670299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7348f-9699-4f4b-9fc7-72f924ea3135 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:07.675406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a6272-0dbe-4db6-99c3-3370b7c8175d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:07.676228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a6272-0dbe-4db6-99c3-3370b7c8175d admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:07.676611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a6272-0dbe-4db6-99c3-3370b7c8175d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:07.677113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a6272-0dbe-4db6-99c3-3370b7c8175d admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:07.677884 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a6272-0dbe-4db6-99c3-3370b7c8175d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:07.680195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a6272-0dbe-4db6-99c3-3370b7c8175d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:07.844163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 960/1954] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:07 2022] POST /identity/v3/projects => generated 385 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:07.846145 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49042cd9-52d0-459a-b793-2ff7e5293d80 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:07.858089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49042cd9-52d0-459a-b793-2ff7e5293d80 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:07.919792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975a234-5537-4818-8b00-4d5bc81cb26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:07.920459 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975a234-5537-4818-8b00-4d5bc81cb26e admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:07.920750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975a234-5537-4818-8b00-4d5bc81cb26e admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:07.921032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975a234-5537-4818-8b00-4d5bc81cb26e admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:07.921669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975a234-5537-4818-8b00-4d5bc81cb26e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:07.923610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975a234-5537-4818-8b00-4d5bc81cb26e admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:08.055960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 995/1955] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:07 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:08.058554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a6272-0dbe-4db6-99c3-3370b7c8175d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:08.071363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a6272-0dbe-4db6-99c3-3370b7c8175d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:08.076823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde6c7-b190-4605-b817-b965648ac2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:08.077455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde6c7-b190-4605-b817-b965648ac2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:08.077747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde6c7-b190-4605-b817-b965648ac2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:08.078027 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde6c7-b190-4605-b817-b965648ac2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:08.078685 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dde6c7-b190-4605-b817-b965648ac2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:08.080975 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dde6c7-b190-4605-b817-b965648ac2d6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:08.093112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 961/1956] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:08.097626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975a234-5537-4818-8b00-4d5bc81cb26e admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:08.108830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975a234-5537-4818-8b00-4d5bc81cb26e admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:08.114506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b21c0-d049-44e6-b004-c9965ae478e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:08.115286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b21c0-d049-44e6-b004-c9965ae478e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:08.115637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b21c0-d049-44e6-b004-c9965ae478e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:08.116018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b21c0-d049-44e6-b004-c9965ae478e9 admin admin] PATH_INFO: `/v3/projects/c128266fce274b958c883e01ee09e346/users/e4c22a8cf27043719752e4f3f1360ad9/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:08.130565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:08.133059 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b21c0-d049-44e6-b004-c9965ae478e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c128266fce274b958c883e01ee09e346, user_id=e4c22a8cf27043719752e4f3f1360ad9, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:08.136050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b21c0-d049-44e6-b004-c9965ae478e9 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:08.247687 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 996/1957] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:08 2022] PUT /identity/v3/projects/c128266fce274b958c883e01ee09e346/users/e4c22a8cf27043719752e4f3f1360ad9/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:08.252280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde6c7-b190-4605-b817-b965648ac2d6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:08.262868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde6c7-b190-4605-b817-b965648ac2d6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:08.339165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c69afe-afc6-48b8-8874-2743c81bd035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:08.339887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69afe-afc6-48b8-8874-2743c81bd035 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:08.341427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69afe-afc6-48b8-8874-2743c81bd035 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:08.341427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69afe-afc6-48b8-8874-2743c81bd035 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:08.341427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c69afe-afc6-48b8-8874-2743c81bd035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:08.342800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c69afe-afc6-48b8-8874-2743c81bd035 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:08.354193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 962/1958] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:51:08.358508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b21c0-d049-44e6-b004-c9965ae478e9 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:08.369768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b21c0-d049-44e6-b004-c9965ae478e9 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:08.375290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca27cb-fb9d-48f6-9d66-34a7d750f340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:08.380275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca27cb-fb9d-48f6-9d66-34a7d750f340 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:08.380275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca27cb-fb9d-48f6-9d66-34a7d750f340 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:08.380275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca27cb-fb9d-48f6-9d66-34a7d750f340 admin admin] PATH_INFO: `/v3/projects/c128266fce274b958c883e01ee09e346/users/e4c22a8cf27043719752e4f3f1360ad9/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:08.380275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca27cb-fb9d-48f6-9d66-34a7d750f340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c128266fce274b958c883e01ee09e346, user_id=e4c22a8cf27043719752e4f3f1360ad9, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:08.381270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca27cb-fb9d-48f6-9d66-34a7d750f340 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:08.392800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 997/1959] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:08 2022] PUT /identity/v3/projects/c128266fce274b958c883e01ee09e346/users/e4c22a8cf27043719752e4f3f1360ad9/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:08.403438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22abe92-2f6a-4579-ad65-029795dd0100 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:08.403847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22abe92-2f6a-4579-ad65-029795dd0100 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:08.404282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22abe92-2f6a-4579-ad65-029795dd0100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:08.437512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b22abe92-2f6a-4579-ad65-029795dd0100 None None] MFA Rules not processed for user `e4c22a8cf27043719752e4f3f1360ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:08.439149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b22abe92-2f6a-4579-ad65-029795dd0100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:08.532389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 963/1960] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:08.547557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca27cb-fb9d-48f6-9d66-34a7d750f340 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:08.557162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9bca27cb-fb9d-48f6-9d66-34a7d750f340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:08.654698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca27cb-fb9d-48f6-9d66-34a7d750f340 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:08.662085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b29dc7-18d4-4f1b-8036-c7382b680ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:08.662085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b29dc7-18d4-4f1b-8036-c7382b680ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:08.662085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b29dc7-18d4-4f1b-8036-c7382b680ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:08.662815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b29dc7-18d4-4f1b-8036-c7382b680ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:08.673950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b29dc7-18d4-4f1b-8036-c7382b680ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:08.675452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b29dc7-18d4-4f1b-8036-c7382b680ee9 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:08.687335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 998/1961] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:08.721102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22abe92-2f6a-4579-ad65-029795dd0100 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:08.809025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22abe92-2f6a-4579-ad65-029795dd0100 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:08.815463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:08.816269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:08.816711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:08.816998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] PATH_INFO: `/v3/users/31ef2c36acad4e1a8e5b96c7ce0f1edd` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:08.818412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31ef2c36acad4e1a8e5b96c7ce0f1edd)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:08.821174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:09.239338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:09.261697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31ef2c36acad4e1a8e5b96c7ce0f1edd'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:09.263651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31ef2c36acad4e1a8e5b96c7ce0f1edd'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:09.279655 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31ef2c36acad4e1a8e5b96c7ce0f1edd'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:09.378772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31ef2c36acad4e1a8e5b96c7ce0f1edd'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:09.387099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31ef2c36acad4e1a8e5b96c7ce0f1edd'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:09.395136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 964/1962] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:08 2022] DELETE /identity/v3/users/31ef2c36acad4e1a8e5b96c7ce0f1edd => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:09.400541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9ba19-a408-4286-8d5c-cda436866711 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:09.401077 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9ba19-a408-4286-8d5c-cda436866711 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:09.401175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9ba19-a408-4286-8d5c-cda436866711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:09.422325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-cfc9ba19-a408-4286-8d5c-cda436866711 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:09.423759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cfc9ba19-a408-4286-8d5c-cda436866711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:09.540731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 999/1963] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:09.558036 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:09.561333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:09.637430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b75be4-0875-46b6-94ba-c10a6537ccf4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:09.642393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf4d8a-a173-4290-9598-60c78f080a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:09.648097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf4d8a-a173-4290-9598-60c78f080a99 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:09.648097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf4d8a-a173-4290-9598-60c78f080a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:09.648097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf4d8a-a173-4290-9598-60c78f080a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:09.657434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf4d8a-a173-4290-9598-60c78f080a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:09.658886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf4d8a-a173-4290-9598-60c78f080a99 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:09.670455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 965/1964] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:10.362042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9ba19-a408-4286-8d5c-cda436866711 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:10.363558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cfc9ba19-a408-4286-8d5c-cda436866711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:10.384133 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9ba19-a408-4286-8d5c-cda436866711 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:10.391272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:10.392258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:10.392705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:10.393715 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] PATH_INFO: `/v3/projects/c3ae52c6349b4cf2990b7acaf7b31266` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:10.395417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3ae52c6349b4cf2990b7acaf7b31266)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:10.397969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:10.538328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3ae52c6349b4cf2990b7acaf7b31266', 'request_id': 'req-235b8f46-5e02-4d92-b356-95fcee2af9bb'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:10.641163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3ae52c6349b4cf2990b7acaf7b31266', 'request_id': 'req-235b8f46-5e02-4d92-b356-95fcee2af9bb'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:10.648082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3ae52c6349b4cf2990b7acaf7b31266', 'request_id': 'req-235b8f46-5e02-4d92-b356-95fcee2af9bb'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:10.658951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] The token cache is being invalidate because project c3ae52c6349b4cf2990b7acaf7b31266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:10.661786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:10.661786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:10.662200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1000/1965] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:10 2022] DELETE /identity/v3/projects/c3ae52c6349b4cf2990b7acaf7b31266 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:10.666120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf4d8a-a173-4290-9598-60c78f080a99 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:10.675959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf4d8a-a173-4290-9598-60c78f080a99 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:10.735354 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ad6dc-07ae-413f-89eb-fee7a8ee8299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:10.736043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad6dc-07ae-413f-89eb-fee7a8ee8299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:10.736331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad6dc-07ae-413f-89eb-fee7a8ee8299 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:10.736607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad6dc-07ae-413f-89eb-fee7a8ee8299 admin admin] PATH_INFO: `/v3/users/e4c22a8cf27043719752e4f3f1360ad9` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:10.737771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ad6dc-07ae-413f-89eb-fee7a8ee8299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c22a8cf27043719752e4f3f1360ad9)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:10.739848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ad6dc-07ae-413f-89eb-fee7a8ee8299 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:11.011544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e71ad6dc-07ae-413f-89eb-fee7a8ee8299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c22a8cf27043719752e4f3f1360ad9'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:11.020370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e71ad6dc-07ae-413f-89eb-fee7a8ee8299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c22a8cf27043719752e4f3f1360ad9'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:11.027929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e71ad6dc-07ae-413f-89eb-fee7a8ee8299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c22a8cf27043719752e4f3f1360ad9'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:11.159532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e71ad6dc-07ae-413f-89eb-fee7a8ee8299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c22a8cf27043719752e4f3f1360ad9'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:11.167928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e71ad6dc-07ae-413f-89eb-fee7a8ee8299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c22a8cf27043719752e4f3f1360ad9'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:11.174743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 966/1966] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:10 2022] DELETE /identity/v3/users/e4c22a8cf27043719752e4f3f1360ad9 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:11.191024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:11.232938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:11.728930 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:11.730368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:11.744798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b8f46-5e02-4d92-b356-95fcee2af9bb admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:11.813094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:11.813759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:11.814051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:11.814337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] PATH_INFO: `/v3/projects/c128266fce274b958c883e01ee09e346` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:11.815576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c128266fce274b958c883e01ee09e346)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:11.817678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:11.948640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c128266fce274b958c883e01ee09e346', 'request_id': 'req-12c45795-ee26-44a9-b6c0-2322ae549548'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:12.043591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c128266fce274b958c883e01ee09e346', 'request_id': 'req-12c45795-ee26-44a9-b6c0-2322ae549548'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:12.051033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c128266fce274b958c883e01ee09e346', 'request_id': 'req-12c45795-ee26-44a9-b6c0-2322ae549548'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:12.061657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] The token cache is being invalidate because project c128266fce274b958c883e01ee09e346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:12.061952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:12.062605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12c45795-ee26-44a9-b6c0-2322ae549548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:12.064134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1001/1967] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:11 2022] DELETE /identity/v3/projects/c128266fce274b958c883e01ee09e346 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:12.069158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcf013-bea2-4bf6-9595-60d48badeaa3 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:12.069637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcf013-bea2-4bf6-9595-60d48badeaa3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:12.069747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcf013-bea2-4bf6-9595-60d48badeaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:12.090791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-4bdcf013-bea2-4bf6-9595-60d48badeaa3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:12.092578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4bdcf013-bea2-4bf6-9595-60d48badeaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:12.151498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 967/1968] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:12.160385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e4858-569b-4294-be25-cf19d7d340dd None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:12.160677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e4858-569b-4294-be25-cf19d7d340dd None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:12.160963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e4858-569b-4294-be25-cf19d7d340dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:12.181347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-962e4858-569b-4294-be25-cf19d7d340dd None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:12.182582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-962e4858-569b-4294-be25-cf19d7d340dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:12.190853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1002/1969] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:12.195434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcf013-bea2-4bf6-9595-60d48badeaa3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:12.205202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcf013-bea2-4bf6-9595-60d48badeaa3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:12.210267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f687d5e-015b-4f77-b490-0e4e1a21796d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:12.210950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f687d5e-015b-4f77-b490-0e4e1a21796d admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:12.211191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f687d5e-015b-4f77-b490-0e4e1a21796d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:12.211471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f687d5e-015b-4f77-b490-0e4e1a21796d admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:12.212277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f687d5e-015b-4f77-b490-0e4e1a21796d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:12.212467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f687d5e-015b-4f77-b490-0e4e1a21796d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:12.214076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f687d5e-015b-4f77-b490-0e4e1a21796d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:12.227770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 968/1970] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:51:12.236416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e4858-569b-4294-be25-cf19d7d340dd None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:12.245837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e4858-569b-4294-be25-cf19d7d340dd None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:12.250710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e27f8-32f1-4eb0-b0e2-623748c33904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:12.251410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e27f8-32f1-4eb0-b0e2-623748c33904 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:12.251765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e27f8-32f1-4eb0-b0e2-623748c33904 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:12.252214 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e27f8-32f1-4eb0-b0e2-623748c33904 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:12.253018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e27f8-32f1-4eb0-b0e2-623748c33904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:12.255151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e27f8-32f1-4eb0-b0e2-623748c33904 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:12.287483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:12.369163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1003/1971] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:12 2022] POST /identity/v3/projects => generated 407 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:12.373607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f687d5e-015b-4f77-b490-0e4e1a21796d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:12.383713 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f687d5e-015b-4f77-b490-0e4e1a21796d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:12.441180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6f8f6-eb1b-4534-9583-10a8e0223945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:12.441805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6f8f6-eb1b-4534-9583-10a8e0223945 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:12.442089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6f8f6-eb1b-4534-9583-10a8e0223945 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:12.442365 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6f8f6-eb1b-4534-9583-10a8e0223945 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:12.442989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6f8f6-eb1b-4534-9583-10a8e0223945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:12.444958 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6f8f6-eb1b-4534-9583-10a8e0223945 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:12.582385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 969/1972] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:12 2022] POST /identity/v3/users => generated 304 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:12.586524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e27f8-32f1-4eb0-b0e2-623748c33904 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:12.603503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e27f8-32f1-4eb0-b0e2-623748c33904 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:12.608874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2df76-d0d0-4b89-8d86-e07c6525986a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:12.609505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2df76-d0d0-4b89-8d86-e07c6525986a admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:12.609792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2df76-d0d0-4b89-8d86-e07c6525986a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:12.610074 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2df76-d0d0-4b89-8d86-e07c6525986a admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:12.610778 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2df76-d0d0-4b89-8d86-e07c6525986a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:12.612450 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2df76-d0d0-4b89-8d86-e07c6525986a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:12.623381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1004/1973] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:51:12.627397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6f8f6-eb1b-4534-9583-10a8e0223945 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:12.637580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6f8f6-eb1b-4534-9583-10a8e0223945 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:12.642957 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb914b-2cc9-41f3-b575-80c235e7d551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:12.648259 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb914b-2cc9-41f3-b575-80c235e7d551 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:12.648259 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb914b-2cc9-41f3-b575-80c235e7d551 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:12.648259 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb914b-2cc9-41f3-b575-80c235e7d551 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:12.648259 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb914b-2cc9-41f3-b575-80c235e7d551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:12.648259 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb914b-2cc9-41f3-b575-80c235e7d551 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:12.659249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 970/1974] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:12.663613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2df76-d0d0-4b89-8d86-e07c6525986a admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:12.674522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2df76-d0d0-4b89-8d86-e07c6525986a admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:12.679131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e1a5b-6a37-49fd-97f7-8957015c8952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:12.679788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e1a5b-6a37-49fd-97f7-8957015c8952 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:12.680122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e1a5b-6a37-49fd-97f7-8957015c8952 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:12.680454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e1a5b-6a37-49fd-97f7-8957015c8952 admin admin] PATH_INFO: `/v3/projects/d1585277ba5040a38159a4c072568c80/users/8664cd686f2e4261841db30a2fa88eac/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:12.695262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e1a5b-6a37-49fd-97f7-8957015c8952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1585277ba5040a38159a4c072568c80, user_id=8664cd686f2e4261841db30a2fa88eac, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:12.697948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e1a5b-6a37-49fd-97f7-8957015c8952 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:12.789554 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1005/1975] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:12 2022] PUT /identity/v3/projects/d1585277ba5040a38159a4c072568c80/users/8664cd686f2e4261841db30a2fa88eac/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:12.800865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05799a5-b656-4d0e-b468-561129afe5e9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:12.801156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05799a5-b656-4d0e-b468-561129afe5e9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:12.801441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05799a5-b656-4d0e-b468-561129afe5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:12.830385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e05799a5-b656-4d0e-b468-561129afe5e9 None None] MFA Rules not processed for user `8664cd686f2e4261841db30a2fa88eac`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:12.834838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e05799a5-b656-4d0e-b468-561129afe5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:12.892964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:12.896409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 971/1976] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:12 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:12.906188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e1a5b-6a37-49fd-97f7-8957015c8952 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:12.974422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e1a5b-6a37-49fd-97f7-8957015c8952 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:12.979208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97921b-eb35-4ba6-a6f1-e079eb94b775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:12.979816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97921b-eb35-4ba6-a6f1-e079eb94b775 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:12.980139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97921b-eb35-4ba6-a6f1-e079eb94b775 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:12.980417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97921b-eb35-4ba6-a6f1-e079eb94b775 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:12.981056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97921b-eb35-4ba6-a6f1-e079eb94b775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:12.982997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97921b-eb35-4ba6-a6f1-e079eb94b775 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:13.112533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1006/1977] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:12 2022] POST /identity/v3/projects => generated 407 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:13.116702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05799a5-b656-4d0e-b468-561129afe5e9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:13.128352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05799a5-b656-4d0e-b468-561129afe5e9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:13.188224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62793973-c88b-4607-adad-54b2a8ff730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:13.188924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62793973-c88b-4607-adad-54b2a8ff730f admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:13.189243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62793973-c88b-4607-adad-54b2a8ff730f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:13.189527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62793973-c88b-4607-adad-54b2a8ff730f admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:13.190178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62793973-c88b-4607-adad-54b2a8ff730f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:13.192355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62793973-c88b-4607-adad-54b2a8ff730f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:13.296608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 972/1978] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:13 2022] POST /identity/v3/users => generated 310 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:51:13.300508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97921b-eb35-4ba6-a6f1-e079eb94b775 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:13.313074 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97921b-eb35-4ba6-a6f1-e079eb94b775 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:13.318099 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877e900-25d4-4ed2-a4ac-8be95db70c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:13.318823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877e900-25d4-4ed2-a4ac-8be95db70c2e admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:13.319112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877e900-25d4-4ed2-a4ac-8be95db70c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:13.319393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877e900-25d4-4ed2-a4ac-8be95db70c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:13.320094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877e900-25d4-4ed2-a4ac-8be95db70c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:13.321742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877e900-25d4-4ed2-a4ac-8be95db70c2e admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:13.332964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1007/1979] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:13.337212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62793973-c88b-4607-adad-54b2a8ff730f admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:13.347327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62793973-c88b-4607-adad-54b2a8ff730f admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:13.349740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:13.352185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f7723-9fe1-4059-8194-a8c23fe036c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:13.352852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7723-9fe1-4059-8194-a8c23fe036c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:13.353138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7723-9fe1-4059-8194-a8c23fe036c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:13.353734 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7723-9fe1-4059-8194-a8c23fe036c6 admin admin] PATH_INFO: `/v3/projects/a99c94f8863d451abd323fef71b4d801/users/661af264b9114d40a4c4e64915bc0dcc/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:13.369418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f7723-9fe1-4059-8194-a8c23fe036c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99c94f8863d451abd323fef71b4d801, user_id=661af264b9114d40a4c4e64915bc0dcc, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:13.372111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f7723-9fe1-4059-8194-a8c23fe036c6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:13.430379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 973/1980] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:13 2022] PUT /identity/v3/projects/a99c94f8863d451abd323fef71b4d801/users/661af264b9114d40a4c4e64915bc0dcc/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:13.434535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877e900-25d4-4ed2-a4ac-8be95db70c2e admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:13.445494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877e900-25d4-4ed2-a4ac-8be95db70c2e admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:13.503299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62b360-033e-4094-b637-061f74bbfca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:13.503944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b360-033e-4094-b637-061f74bbfca2 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:13.504234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b360-033e-4094-b637-061f74bbfca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:13.504516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b360-033e-4094-b637-061f74bbfca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:13.505162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62b360-033e-4094-b637-061f74bbfca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:13.506767 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62b360-033e-4094-b637-061f74bbfca2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:13.516397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1008/1981] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:13.520698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f7723-9fe1-4059-8194-a8c23fe036c6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:13.531412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f7723-9fe1-4059-8194-a8c23fe036c6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:13.537742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49270bff-8447-4a02-a5c5-bb61ed185c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:13.538428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49270bff-8447-4a02-a5c5-bb61ed185c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:13.538716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49270bff-8447-4a02-a5c5-bb61ed185c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:13.538984 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49270bff-8447-4a02-a5c5-bb61ed185c29 admin admin] PATH_INFO: `/v3/projects/a99c94f8863d451abd323fef71b4d801/users/661af264b9114d40a4c4e64915bc0dcc/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:13.541027 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49270bff-8447-4a02-a5c5-bb61ed185c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99c94f8863d451abd323fef71b4d801, user_id=661af264b9114d40a4c4e64915bc0dcc, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:13.543601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49270bff-8447-4a02-a5c5-bb61ed185c29 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:13.555896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 974/1982] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:13 2022] PUT /identity/v3/projects/a99c94f8863d451abd323fef71b4d801/users/661af264b9114d40a4c4e64915bc0dcc/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:13.566496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef41d6e-639f-4cf8-a0ea-2ff36090fe3d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:13.566790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef41d6e-639f-4cf8-a0ea-2ff36090fe3d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:13.567069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef41d6e-639f-4cf8-a0ea-2ff36090fe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:13.603585 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3ef41d6e-639f-4cf8-a0ea-2ff36090fe3d None None] MFA Rules not processed for user `661af264b9114d40a4c4e64915bc0dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:13.605115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3ef41d6e-639f-4cf8-a0ea-2ff36090fe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:13.739817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1009/1983] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:13 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:13.752991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49270bff-8447-4a02-a5c5-bb61ed185c29 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:13.754739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-49270bff-8447-4a02-a5c5-bb61ed185c29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:13.842287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49270bff-8447-4a02-a5c5-bb61ed185c29 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:13.846901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1c643-7269-40dd-b087-a96868ee525f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:13.851850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1c643-7269-40dd-b087-a96868ee525f service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:13.856465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1c643-7269-40dd-b087-a96868ee525f service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:13.856943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1c643-7269-40dd-b087-a96868ee525f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:13.867909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1c643-7269-40dd-b087-a96868ee525f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:13.870071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1c643-7269-40dd-b087-a96868ee525f service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:13.880705 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 975/1984] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:13 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:13.906900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:14.573308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef41d6e-639f-4cf8-a0ea-2ff36090fe3d None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:14.575042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3ef41d6e-639f-4cf8-a0ea-2ff36090fe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:14.593695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:14.668122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef41d6e-639f-4cf8-a0ea-2ff36090fe3d None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:14.674517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:14.675564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:14.676104 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:14.676659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] PATH_INFO: `/v3/users/a430ea6c831b4f5ab6a913ae240ae88d` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:14.678049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a430ea6c831b4f5ab6a913ae240ae88d)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:14.680274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:14.722783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1c643-7269-40dd-b087-a96868ee525f service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:14.724263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5cc1c643-7269-40dd-b087-a96868ee525f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:14.736192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1c643-7269-40dd-b087-a96868ee525f service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:14.739970 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4f99a-939b-4058-973f-7ca0bbc7b976 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:14.740612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4f99a-939b-4058-973f-7ca0bbc7b976 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:14.740901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4f99a-939b-4058-973f-7ca0bbc7b976 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:14.741181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4f99a-939b-4058-973f-7ca0bbc7b976 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:14.805738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4f99a-939b-4058-973f-7ca0bbc7b976 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:14.807218 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4f99a-939b-4058-973f-7ca0bbc7b976 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:14.817202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 976/1985] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:14 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:14.923615 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a430ea6c831b4f5ab6a913ae240ae88d'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:14.949444 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a430ea6c831b4f5ab6a913ae240ae88d'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:14.950749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a430ea6c831b4f5ab6a913ae240ae88d'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:14.955119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a430ea6c831b4f5ab6a913ae240ae88d'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:15.085733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a430ea6c831b4f5ab6a913ae240ae88d'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:15.099835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1010/1986] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:14 2022] DELETE /identity/v3/users/a430ea6c831b4f5ab6a913ae240ae88d => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:15.104319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc38ba7-cb72-45ea-a526-d57e1c955d5a None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:15.104702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc38ba7-cb72-45ea-a526-d57e1c955d5a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:15.105065 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc38ba7-cb72-45ea-a526-d57e1c955d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:15.127112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7fc38ba7-cb72-45ea-a526-d57e1c955d5a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:15.128465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7fc38ba7-cb72-45ea-a526-d57e1c955d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:15.196413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 977/1987] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:15.206329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:15.208090 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:15.266859 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a66bf-5bdc-4e87-9789-366b92afaa98 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:15.275278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2dc81-a02a-4c86-b0a3-0460e4e831be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:15.276003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2dc81-a02a-4c86-b0a3-0460e4e831be service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:15.277637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2dc81-a02a-4c86-b0a3-0460e4e831be service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:15.277637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2dc81-a02a-4c86-b0a3-0460e4e831be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:15.290489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2dc81-a02a-4c86-b0a3-0460e4e831be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:15.292396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2dc81-a02a-4c86-b0a3-0460e4e831be service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:15.305445 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1011/1988] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:15.384303 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:15.820051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc38ba7-cb72-45ea-a526-d57e1c955d5a None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:15.821597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7fc38ba7-cb72-45ea-a526-d57e1c955d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:15.836233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc38ba7-cb72-45ea-a526-d57e1c955d5a None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:15.839443 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:15.840127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:15.840416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:15.840753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] PATH_INFO: `/v3/projects/6120fe64de3946849ac8204aa6828074` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:15.841881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6120fe64de3946849ac8204aa6828074)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:15.843988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:15.971543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6120fe64de3946849ac8204aa6828074', 'request_id': 'req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:15.977013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6120fe64de3946849ac8204aa6828074', 'request_id': 'req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:15.983446 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6120fe64de3946849ac8204aa6828074', 'request_id': 'req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:15.987180 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:16.098146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] The token cache is being invalidate because project 6120fe64de3946849ac8204aa6828074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:16.098448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:16.099744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bca9b9f-376c-43a5-9a62-3dff8ff080f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:16.101483 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 978/1989] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:15 2022] DELETE /identity/v3/projects/6120fe64de3946849ac8204aa6828074 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:16.107071 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d101d-010c-4262-8072-f30173491134 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.110378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d101d-010c-4262-8072-f30173491134 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.110378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d101d-010c-4262-8072-f30173491134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.131243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-c00d101d-010c-4262-8072-f30173491134 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:16.132775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c00d101d-010c-4262-8072-f30173491134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:16.196923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1012/1990] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:16.204633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97426686-7a4c-4184-975d-f4f214aa3ff9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.205060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97426686-7a4c-4184-975d-f4f214aa3ff9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.205173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97426686-7a4c-4184-975d-f4f214aa3ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.225609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-97426686-7a4c-4184-975d-f4f214aa3ff9 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:16.226777 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-97426686-7a4c-4184-975d-f4f214aa3ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:16.236295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 979/1991] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:16.240809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d101d-010c-4262-8072-f30173491134 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:16.252007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d101d-010c-4262-8072-f30173491134 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:16.256961 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5dc4a-c9ff-4630-8494-fa76a2756623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:16.257552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5dc4a-c9ff-4630-8494-fa76a2756623 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.257837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5dc4a-c9ff-4630-8494-fa76a2756623 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.258685 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5dc4a-c9ff-4630-8494-fa76a2756623 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.258801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5dc4a-c9ff-4630-8494-fa76a2756623 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:16.259109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5dc4a-c9ff-4630-8494-fa76a2756623 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:16.260826 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5dc4a-c9ff-4630-8494-fa76a2756623 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:16.274233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1013/1992] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:51:16.278798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97426686-7a4c-4184-975d-f4f214aa3ff9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:16.288664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97426686-7a4c-4184-975d-f4f214aa3ff9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:16.294675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab41817-eeb1-4604-8a43-96c61459cb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:16.297013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab41817-eeb1-4604-8a43-96c61459cb8d admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.297013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab41817-eeb1-4604-8a43-96c61459cb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.297013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab41817-eeb1-4604-8a43-96c61459cb8d admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.297339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab41817-eeb1-4604-8a43-96c61459cb8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:16.299282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab41817-eeb1-4604-8a43-96c61459cb8d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:16.423790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 980/1993] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:16 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:16.427842 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5dc4a-c9ff-4630-8494-fa76a2756623 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:16.439196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5dc4a-c9ff-4630-8494-fa76a2756623 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:16.501087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2327e7f-3423-4a1f-85c7-b58108a9c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:16.501649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2327e7f-3423-4a1f-85c7-b58108a9c6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.501936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2327e7f-3423-4a1f-85c7-b58108a9c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.502212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2327e7f-3423-4a1f-85c7-b58108a9c6d1 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.502841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2327e7f-3423-4a1f-85c7-b58108a9c6d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:16.504861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2327e7f-3423-4a1f-85c7-b58108a9c6d1 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:16.647799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1014/1994] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:16 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:51:16.649699 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab41817-eeb1-4604-8a43-96c61459cb8d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:16.661066 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab41817-eeb1-4604-8a43-96c61459cb8d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:16.665982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaf091-3ecd-467c-a4cc-f9d32df71df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:16.668599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaf091-3ecd-467c-a4cc-f9d32df71df2 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.668599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaf091-3ecd-467c-a4cc-f9d32df71df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.668599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaf091-3ecd-467c-a4cc-f9d32df71df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.668599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaf091-3ecd-467c-a4cc-f9d32df71df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:16.671531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaf091-3ecd-467c-a4cc-f9d32df71df2 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:16.682672 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 981/1995] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:16.687270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2327e7f-3423-4a1f-85c7-b58108a9c6d1 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:16.697881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2327e7f-3423-4a1f-85c7-b58108a9c6d1 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:16.702859 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15817dcc-0b33-4bba-8240-0e09016a8b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:16.703499 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15817dcc-0b33-4bba-8240-0e09016a8b7e admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.703789 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15817dcc-0b33-4bba-8240-0e09016a8b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.704130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15817dcc-0b33-4bba-8240-0e09016a8b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.704912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15817dcc-0b33-4bba-8240-0e09016a8b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:16.706987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15817dcc-0b33-4bba-8240-0e09016a8b7e admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:16.724557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1015/1996] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:16.726883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaf091-3ecd-467c-a4cc-f9d32df71df2 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:16.737732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaf091-3ecd-467c-a4cc-f9d32df71df2 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:16.742622 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a2f4-bf03-4222-bc66-9da34c601a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:16.743237 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a2f4-bf03-4222-bc66-9da34c601a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.743526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a2f4-bf03-4222-bc66-9da34c601a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.743804 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a2f4-bf03-4222-bc66-9da34c601a01 admin admin] PATH_INFO: `/v3/projects/016b0c382e92427cbfe7a99c1104caa1/users/d07de326711142f982c75d6281a510e6/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.766153 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000a2f4-bf03-4222-bc66-9da34c601a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=016b0c382e92427cbfe7a99c1104caa1, user_id=d07de326711142f982c75d6281a510e6, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:16.769605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000a2f4-bf03-4222-bc66-9da34c601a01 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:16.875915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:16.922221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 982/1997] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:16 2022] PUT /identity/v3/projects/016b0c382e92427cbfe7a99c1104caa1/users/d07de326711142f982c75d6281a510e6/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:16.933824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92e49-2500-4102-8efc-36a0b91ad5a8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.934115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92e49-2500-4102-8efc-36a0b91ad5a8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.937224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92e49-2500-4102-8efc-36a0b91ad5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.963526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-5fd92e49-2500-4102-8efc-36a0b91ad5a8 None None] MFA Rules not processed for user `d07de326711142f982c75d6281a510e6`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:16.964807 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5fd92e49-2500-4102-8efc-36a0b91ad5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:17.051140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1016/1998] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:17.109837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a2f4-bf03-4222-bc66-9da34c601a01 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:17.111565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3000a2f4-bf03-4222-bc66-9da34c601a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:17.170969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a2f4-bf03-4222-bc66-9da34c601a01 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:17.174730 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e75018-7dc8-4a78-ae35-027a2e702610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:17.175363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e75018-7dc8-4a78-ae35-027a2e702610 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:17.175650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e75018-7dc8-4a78-ae35-027a2e702610 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:17.175964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e75018-7dc8-4a78-ae35-027a2e702610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:17.187691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e75018-7dc8-4a78-ae35-027a2e702610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:17.194364 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e75018-7dc8-4a78-ae35-027a2e702610 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:17.199107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 983/1999] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:18.404677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:18.507764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:18.672270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd92e49-2500-4102-8efc-36a0b91ad5a8 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:18.673740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-5fd92e49-2500-4102-8efc-36a0b91ad5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:18.758813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd92e49-2500-4102-8efc-36a0b91ad5a8 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:18.763378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:18.764044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:18.764332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:18.764613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] PATH_INFO: `/v3/users/62c9133e061841aea7f20e00dacab5ef` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:18.766390 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=62c9133e061841aea7f20e00dacab5ef)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:18.768360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:19.055739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62c9133e061841aea7f20e00dacab5ef'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:19.068460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62c9133e061841aea7f20e00dacab5ef'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:19.069132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62c9133e061841aea7f20e00dacab5ef'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:19.069786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62c9133e061841aea7f20e00dacab5ef'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:19.127393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:19.155575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62c9133e061841aea7f20e00dacab5ef'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:19.169032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1017/2000] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:18 2022] DELETE /identity/v3/users/62c9133e061841aea7f20e00dacab5ef => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:19.174031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad107d7-db12-4a9e-8435-54d053cc2598 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:19.174330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad107d7-db12-4a9e-8435-54d053cc2598 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:19.174613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad107d7-db12-4a9e-8435-54d053cc2598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:19.194694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-5ad107d7-db12-4a9e-8435-54d053cc2598 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:19.196122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5ad107d7-db12-4a9e-8435-54d053cc2598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:19.258925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 984/2001] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:19.268736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:19.270171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:19.335993 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb00e60-1a85-4824-b527-97bb38ff23ad admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:19.341739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f4f41-f014-4b7e-929a-f07c8aefff99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:19.342775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f4f41-f014-4b7e-929a-f07c8aefff99 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:19.343373 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f4f41-f014-4b7e-929a-f07c8aefff99 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:19.343943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f4f41-f014-4b7e-929a-f07c8aefff99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:19.353446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f4f41-f014-4b7e-929a-f07c8aefff99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:19.354913 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f4f41-f014-4b7e-929a-f07c8aefff99 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:19.364801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1018/2002] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:19.627596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:19.701542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:19.801170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad107d7-db12-4a9e-8435-54d053cc2598 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:19.808020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5ad107d7-db12-4a9e-8435-54d053cc2598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:19.819294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad107d7-db12-4a9e-8435-54d053cc2598 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:19.825500 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:19.826439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:19.826615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:19.827050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] PATH_INFO: `/v3/projects/659f383d58c240efb3e15789e4cf8942` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:19.828944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] RBAC: Authorizing `identity:delete_project(project_id=659f383d58c240efb3e15789e4cf8942)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:19.830992 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:19.942901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659f383d58c240efb3e15789e4cf8942', 'request_id': 'req-9a46639e-b439-4594-8844-a0fcf48e0ccf'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:19.948518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659f383d58c240efb3e15789e4cf8942', 'request_id': 'req-9a46639e-b439-4594-8844-a0fcf48e0ccf'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:19.955752 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659f383d58c240efb3e15789e4cf8942', 'request_id': 'req-9a46639e-b439-4594-8844-a0fcf48e0ccf'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:20.089468 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] The token cache is being invalidate because project 659f383d58c240efb3e15789e4cf8942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:20.089761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:20.090390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:20.091948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 985/2003] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:19 2022] DELETE /identity/v3/projects/659f383d58c240efb3e15789e4cf8942 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:20.096722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f4f41-f014-4b7e-929a-f07c8aefff99 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:20.106825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f4f41-f014-4b7e-929a-f07c8aefff99 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:20.165165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ca15d-f469-418b-8a43-0879be346f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:20.165904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ca15d-f469-418b-8a43-0879be346f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:20.166190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ca15d-f469-418b-8a43-0879be346f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:20.166467 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ca15d-f469-418b-8a43-0879be346f6b admin admin] PATH_INFO: `/v3/users/5072aaa7c9494340ac8e253dbf605fdd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:20.167611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ca15d-f469-418b-8a43-0879be346f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5072aaa7c9494340ac8e253dbf605fdd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:20.169664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ca15d-f469-418b-8a43-0879be346f6b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:20.366525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-177ca15d-f469-418b-8a43-0879be346f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5072aaa7c9494340ac8e253dbf605fdd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:20.378672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-177ca15d-f469-418b-8a43-0879be346f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5072aaa7c9494340ac8e253dbf605fdd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:20.379352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-177ca15d-f469-418b-8a43-0879be346f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5072aaa7c9494340ac8e253dbf605fdd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:20.380097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-177ca15d-f469-418b-8a43-0879be346f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5072aaa7c9494340ac8e253dbf605fdd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:20.495050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-177ca15d-f469-418b-8a43-0879be346f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5072aaa7c9494340ac8e253dbf605fdd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:20.511328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1019/2004] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:20 2022] DELETE /identity/v3/users/5072aaa7c9494340ac8e253dbf605fdd => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:21.030132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:21.032131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:21.045389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46639e-b439-4594-8844-a0fcf48e0ccf admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:21.050196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:21.102577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:21.133050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:21.133796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:21.134221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:21.134641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] PATH_INFO: `/v3/projects/d2a1d075a06349158c04679a85241acf` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:21.136379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2a1d075a06349158c04679a85241acf)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:21.139454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:21.250216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2a1d075a06349158c04679a85241acf', 'request_id': 'req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:21.255424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2a1d075a06349158c04679a85241acf', 'request_id': 'req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:21.261568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2a1d075a06349158c04679a85241acf', 'request_id': 'req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:21.352335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] The token cache is being invalidate because project d2a1d075a06349158c04679a85241acf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:21.352758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:21.354844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-21a4a3dd-f3f1-4723-b864-05cf8c9a8aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:21.356438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 986/2005] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:21 2022] DELETE /identity/v3/projects/d2a1d075a06349158c04679a85241acf => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:21.361992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa915b-1c73-49b4-9172-f61a4f2d607f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:21.362281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa915b-1c73-49b4-9172-f61a4f2d607f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:21.362558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa915b-1c73-49b4-9172-f61a4f2d607f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:21.389294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f0aa915b-1c73-49b4-9172-f61a4f2d607f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:21.391306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f0aa915b-1c73-49b4-9172-f61a4f2d607f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:21.451923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1020/2006] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:21.460540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba082bd-f682-4468-974d-4d01c107c86b None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:21.460848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba082bd-f682-4468-974d-4d01c107c86b None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:21.461131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba082bd-f682-4468-974d-4d01c107c86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:21.481282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-dba082bd-f682-4468-974d-4d01c107c86b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:21.482515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-dba082bd-f682-4468-974d-4d01c107c86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:21.490868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 987/2007] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:21.495279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa915b-1c73-49b4-9172-f61a4f2d607f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:21.507170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa915b-1c73-49b4-9172-f61a4f2d607f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:21.509817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db676511-ec17-4af1-8dc9-7abb00a4fb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:21.510423 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db676511-ec17-4af1-8dc9-7abb00a4fb04 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:21.510712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db676511-ec17-4af1-8dc9-7abb00a4fb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:21.510990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db676511-ec17-4af1-8dc9-7abb00a4fb04 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:21.511681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db676511-ec17-4af1-8dc9-7abb00a4fb04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:21.512049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db676511-ec17-4af1-8dc9-7abb00a4fb04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:21.513648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db676511-ec17-4af1-8dc9-7abb00a4fb04 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:21.525987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1021/2008] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:51:21.530483 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba082bd-f682-4468-974d-4d01c107c86b None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:21.540734 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba082bd-f682-4468-974d-4d01c107c86b None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:21.546165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfc3e8-efaf-4901-9444-ec6cdaf2bf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:21.546988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfc3e8-efaf-4901-9444-ec6cdaf2bf98 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:21.551974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfc3e8-efaf-4901-9444-ec6cdaf2bf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:21.551974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfc3e8-efaf-4901-9444-ec6cdaf2bf98 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:21.551974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfc3e8-efaf-4901-9444-ec6cdaf2bf98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:21.551974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfc3e8-efaf-4901-9444-ec6cdaf2bf98 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:21.646662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 988/2009] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:21 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:21.650856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db676511-ec17-4af1-8dc9-7abb00a4fb04 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:21.662052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db676511-ec17-4af1-8dc9-7abb00a4fb04 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:21.724986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d9aea-d2f3-4b27-8a7d-dd53ce69fa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:21.725629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d9aea-d2f3-4b27-8a7d-dd53ce69fa42 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:21.725921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d9aea-d2f3-4b27-8a7d-dd53ce69fa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:21.726202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d9aea-d2f3-4b27-8a7d-dd53ce69fa42 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:21.726866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d9aea-d2f3-4b27-8a7d-dd53ce69fa42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:21.728954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d9aea-d2f3-4b27-8a7d-dd53ce69fa42 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:21.817270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1022/2010] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:21 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:21.821665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfc3e8-efaf-4901-9444-ec6cdaf2bf98 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:21.833297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfc3e8-efaf-4901-9444-ec6cdaf2bf98 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:21.838167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84207e-3b33-403b-a49e-29ba4eeebd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:21.838783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84207e-3b33-403b-a49e-29ba4eeebd7f admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:21.839070 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84207e-3b33-403b-a49e-29ba4eeebd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:21.839349 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84207e-3b33-403b-a49e-29ba4eeebd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:21.840043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84207e-3b33-403b-a49e-29ba4eeebd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:21.841661 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84207e-3b33-403b-a49e-29ba4eeebd7f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:21.852116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 989/2011] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:21.856280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d9aea-d2f3-4b27-8a7d-dd53ce69fa42 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:21.867551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d9aea-d2f3-4b27-8a7d-dd53ce69fa42 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:21.872726 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab48c41-606b-480a-a681-aa617aa8c00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:21.873343 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab48c41-606b-480a-a681-aa617aa8c00c admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:21.880600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab48c41-606b-480a-a681-aa617aa8c00c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:21.880600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab48c41-606b-480a-a681-aa617aa8c00c admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:21.880600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab48c41-606b-480a-a681-aa617aa8c00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:21.880600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab48c41-606b-480a-a681-aa617aa8c00c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:21.887978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1023/2012] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:21.893008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84207e-3b33-403b-a49e-29ba4eeebd7f admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:21.903178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84207e-3b33-403b-a49e-29ba4eeebd7f admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:21.908009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd94050-38b2-4df5-ac24-3256b473b305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:21.908689 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd94050-38b2-4df5-ac24-3256b473b305 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:21.908970 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd94050-38b2-4df5-ac24-3256b473b305 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:21.909254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd94050-38b2-4df5-ac24-3256b473b305 admin admin] PATH_INFO: `/v3/projects/c0bdeb172d0145ba9804c7c10b54bdf0/users/5708a323e4ef443a9efabdbcc0425316/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:21.924098 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd94050-38b2-4df5-ac24-3256b473b305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0bdeb172d0145ba9804c7c10b54bdf0, user_id=5708a323e4ef443a9efabdbcc0425316, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:21.926732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd94050-38b2-4df5-ac24-3256b473b305 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:21.941620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:22.074884 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 990/2013] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:21 2022] PUT /identity/v3/projects/c0bdeb172d0145ba9804c7c10b54bdf0/users/5708a323e4ef443a9efabdbcc0425316/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:22.081849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2ba08-0869-4f48-b4c9-8dc3f12c1560 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:22.082370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2ba08-0869-4f48-b4c9-8dc3f12c1560 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:22.082661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2ba08-0869-4f48-b4c9-8dc3f12c1560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:22.133796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-dfb2ba08-0869-4f48-b4c9-8dc3f12c1560 None None] MFA Rules not processed for user `5708a323e4ef443a9efabdbcc0425316`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:22.133796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-dfb2ba08-0869-4f48-b4c9-8dc3f12c1560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:22.203941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1024/2014] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:22.207003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd94050-38b2-4df5-ac24-3256b473b305 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:22.272725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd94050-38b2-4df5-ac24-3256b473b305 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:22.277543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00152a75-5646-42b7-8368-040048e7106b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:22.278164 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00152a75-5646-42b7-8368-040048e7106b admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:22.278452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00152a75-5646-42b7-8368-040048e7106b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:22.278761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00152a75-5646-42b7-8368-040048e7106b admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:22.279396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00152a75-5646-42b7-8368-040048e7106b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:22.281389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00152a75-5646-42b7-8368-040048e7106b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:22.382809 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 991/2015] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:22 2022] POST /identity/v3/projects => generated 363 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:22.386073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2ba08-0869-4f48-b4c9-8dc3f12c1560 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:22.397314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2ba08-0869-4f48-b4c9-8dc3f12c1560 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:22.459271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdea65-4474-4a36-b351-29c37a87bac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:22.459894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdea65-4474-4a36-b351-29c37a87bac4 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:22.460196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdea65-4474-4a36-b351-29c37a87bac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:22.460471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdea65-4474-4a36-b351-29c37a87bac4 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:22.461093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdea65-4474-4a36-b351-29c37a87bac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:22.463202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdea65-4474-4a36-b351-29c37a87bac4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:22.470398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:22.619317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1025/2016] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:22 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:22.623669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00152a75-5646-42b7-8368-040048e7106b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:22.634487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00152a75-5646-42b7-8368-040048e7106b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:22.639429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d1eab-b97f-4ad8-aef1-4d6cba117803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:22.640135 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d1eab-b97f-4ad8-aef1-4d6cba117803 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:22.640418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d1eab-b97f-4ad8-aef1-4d6cba117803 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:22.640695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d1eab-b97f-4ad8-aef1-4d6cba117803 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:22.641350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d1eab-b97f-4ad8-aef1-4d6cba117803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:22.642958 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d1eab-b97f-4ad8-aef1-4d6cba117803 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:22.654332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 992/2017] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:51:22.659111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdea65-4474-4a36-b351-29c37a87bac4 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:22.669736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdea65-4474-4a36-b351-29c37a87bac4 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:22.675162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0c5c3-52e3-4911-baa5-17dd2f69f0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:22.677856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0c5c3-52e3-4911-baa5-17dd2f69f0e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:22.677856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0c5c3-52e3-4911-baa5-17dd2f69f0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:22.677856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0c5c3-52e3-4911-baa5-17dd2f69f0e2 admin admin] PATH_INFO: `/v3/projects/12ba31dd37c9447b82eb18c66c14f980/users/dd5281d87fda40719df40414a316afd5/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:22.692842 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0c5c3-52e3-4911-baa5-17dd2f69f0e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ba31dd37c9447b82eb18c66c14f980, user_id=dd5281d87fda40719df40414a316afd5, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:22.695607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0c5c3-52e3-4911-baa5-17dd2f69f0e2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:22.782994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1026/2018] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:22 2022] PUT /identity/v3/projects/12ba31dd37c9447b82eb18c66c14f980/users/dd5281d87fda40719df40414a316afd5/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:22.787270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d1eab-b97f-4ad8-aef1-4d6cba117803 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:22.800225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d1eab-b97f-4ad8-aef1-4d6cba117803 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:22.867317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd20ca-eb0a-47ff-926e-42aed56d5417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:22.867974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd20ca-eb0a-47ff-926e-42aed56d5417 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:22.868321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd20ca-eb0a-47ff-926e-42aed56d5417 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:22.868604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd20ca-eb0a-47ff-926e-42aed56d5417 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:22.869375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd20ca-eb0a-47ff-926e-42aed56d5417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:22.871003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd20ca-eb0a-47ff-926e-42aed56d5417 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:22.881517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 993/2019] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:22.886992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0c5c3-52e3-4911-baa5-17dd2f69f0e2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:22.898920 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0c5c3-52e3-4911-baa5-17dd2f69f0e2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:22.904192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd7d6d-3eb1-45a5-a3f2-48ca7d425005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:22.904932 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd7d6d-3eb1-45a5-a3f2-48ca7d425005 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:22.906069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd7d6d-3eb1-45a5-a3f2-48ca7d425005 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:22.906359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd7d6d-3eb1-45a5-a3f2-48ca7d425005 admin admin] PATH_INFO: `/v3/projects/12ba31dd37c9447b82eb18c66c14f980/users/dd5281d87fda40719df40414a316afd5/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:22.908362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd7d6d-3eb1-45a5-a3f2-48ca7d425005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ba31dd37c9447b82eb18c66c14f980, user_id=dd5281d87fda40719df40414a316afd5, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:22.911365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd7d6d-3eb1-45a5-a3f2-48ca7d425005 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:22.922471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1027/2020] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:22 2022] PUT /identity/v3/projects/12ba31dd37c9447b82eb18c66c14f980/users/dd5281d87fda40719df40414a316afd5/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:22.933953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7b3a-004e-4219-baf8-97e320b8da34 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:22.934285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7b3a-004e-4219-baf8-97e320b8da34 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:22.934570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7b3a-004e-4219-baf8-97e320b8da34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:22.964371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c07c7b3a-004e-4219-baf8-97e320b8da34 None None] MFA Rules not processed for user `dd5281d87fda40719df40414a316afd5`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:22.965612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c07c7b3a-004e-4219-baf8-97e320b8da34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:23.027746 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:23.037625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 994/2021] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:23.049693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd7d6d-3eb1-45a5-a3f2-48ca7d425005 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:23.051706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-dadd7d6d-3eb1-45a5-a3f2-48ca7d425005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:23.115469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd7d6d-3eb1-45a5-a3f2-48ca7d425005 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:23.119927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39126870-7757-4792-be6c-8281af17127c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:23.121217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39126870-7757-4792-be6c-8281af17127c service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:23.121217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39126870-7757-4792-be6c-8281af17127c service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:23.121217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39126870-7757-4792-be6c-8281af17127c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:23.131119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39126870-7757-4792-be6c-8281af17127c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:23.132532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39126870-7757-4792-be6c-8281af17127c service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:23.143241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1028/2022] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:23.147115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:23.430981 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:23.456515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7b3a-004e-4219-baf8-97e320b8da34 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:23.464411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7b3a-004e-4219-baf8-97e320b8da34 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:23.468483 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de36d1-d192-465f-aa4e-4f1125308b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:23.469117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de36d1-d192-465f-aa4e-4f1125308b21 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:23.469427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de36d1-d192-465f-aa4e-4f1125308b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:23.469709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de36d1-d192-465f-aa4e-4f1125308b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:23.544449 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de36d1-d192-465f-aa4e-4f1125308b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:23.545926 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de36d1-d192-465f-aa4e-4f1125308b21 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:23.555577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 995/2023] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:23.850643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:24.061268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39126870-7757-4792-be6c-8281af17127c service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:24.063072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-39126870-7757-4792-be6c-8281af17127c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:24.136759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39126870-7757-4792-be6c-8281af17127c service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:24.142028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:24.142671 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:24.142960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:24.143241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] PATH_INFO: `/v3/users/437f2e87a8574262a19a15c5034cc949` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:24.144405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] RBAC: Authorizing `identity:delete_user(user_id=437f2e87a8574262a19a15c5034cc949)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:24.146356 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:24.375023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437f2e87a8574262a19a15c5034cc949'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:24.386814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437f2e87a8574262a19a15c5034cc949'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:24.387529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437f2e87a8574262a19a15c5034cc949'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:24.388171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437f2e87a8574262a19a15c5034cc949'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:24.504832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437f2e87a8574262a19a15c5034cc949'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:24.524881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1029/2024] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:24 2022] DELETE /identity/v3/users/437f2e87a8574262a19a15c5034cc949 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:24.530465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d90dc-b5b3-4e6f-890f-2027c7246efa None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:24.531515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d90dc-b5b3-4e6f-890f-2027c7246efa None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:24.531890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d90dc-b5b3-4e6f-890f-2027c7246efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:24.558315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ed7d90dc-b5b3-4e6f-890f-2027c7246efa None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:24.559592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ed7d90dc-b5b3-4e6f-890f-2027c7246efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:24.639732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 996/2025] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:24.650415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:24.652003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:24.714409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a4e8f-160c-48a5-803b-1f29b7f4e75e admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:24.718062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535eb72d-eb6a-4b04-aaee-0c190db9b811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:24.718667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535eb72d-eb6a-4b04-aaee-0c190db9b811 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:24.718950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535eb72d-eb6a-4b04-aaee-0c190db9b811 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:24.719248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535eb72d-eb6a-4b04-aaee-0c190db9b811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:24.729074 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535eb72d-eb6a-4b04-aaee-0c190db9b811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:24.730576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535eb72d-eb6a-4b04-aaee-0c190db9b811 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:24.741599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1030/2026] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:24.853680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:25.134582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d90dc-b5b3-4e6f-890f-2027c7246efa None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:25.137309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ed7d90dc-b5b3-4e6f-890f-2027c7246efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:25.148384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d90dc-b5b3-4e6f-890f-2027c7246efa None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:25.153555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518698e-f847-4dc4-bf8a-c31de54793e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:25.154220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518698e-f847-4dc4-bf8a-c31de54793e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:25.154509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518698e-f847-4dc4-bf8a-c31de54793e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:25.154790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518698e-f847-4dc4-bf8a-c31de54793e2 admin admin] PATH_INFO: `/v3/users/d07de326711142f982c75d6281a510e6` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:25.156888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518698e-f847-4dc4-bf8a-c31de54793e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d07de326711142f982c75d6281a510e6)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:25.158947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518698e-f847-4dc4-bf8a-c31de54793e2 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:25.290923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535eb72d-eb6a-4b04-aaee-0c190db9b811 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:25.292396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-535eb72d-eb6a-4b04-aaee-0c190db9b811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:25.306301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535eb72d-eb6a-4b04-aaee-0c190db9b811 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:25.312931 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:25.314853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:25.315304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:25.315731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] PATH_INFO: `/v3/projects/20db06d2caf84ad2957fbdf0acb7deb0` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:25.322621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20db06d2caf84ad2957fbdf0acb7deb0)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:25.322621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:25.389921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:25.415192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b518698e-f847-4dc4-bf8a-c31de54793e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd07de326711142f982c75d6281a510e6'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:25.416397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b518698e-f847-4dc4-bf8a-c31de54793e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd07de326711142f982c75d6281a510e6'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:25.431205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b518698e-f847-4dc4-bf8a-c31de54793e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd07de326711142f982c75d6281a510e6'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:25.486097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20db06d2caf84ad2957fbdf0acb7deb0', 'request_id': 'req-530eb522-6899-491f-bd5e-5bd110792ed3'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:25.518627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b518698e-f847-4dc4-bf8a-c31de54793e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd07de326711142f982c75d6281a510e6'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:25.527696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b518698e-f847-4dc4-bf8a-c31de54793e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd07de326711142f982c75d6281a510e6'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:25.534886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 997/2027] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:25 2022] DELETE /identity/v3/users/d07de326711142f982c75d6281a510e6 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:25.539595 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf05dcc-29d1-4395-bd20-3eafc36b07a5 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:25.539960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf05dcc-29d1-4395-bd20-3eafc36b07a5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:25.540193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf05dcc-29d1-4395-bd20-3eafc36b07a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:25.563025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-dcf05dcc-29d1-4395-bd20-3eafc36b07a5 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:25.564369 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-dcf05dcc-29d1-4395-bd20-3eafc36b07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:25.611781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20db06d2caf84ad2957fbdf0acb7deb0', 'request_id': 'req-530eb522-6899-491f-bd5e-5bd110792ed3'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:25.625581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20db06d2caf84ad2957fbdf0acb7deb0', 'request_id': 'req-530eb522-6899-491f-bd5e-5bd110792ed3'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:25.636482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] The token cache is being invalidate because project 20db06d2caf84ad2957fbdf0acb7deb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:25.636794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:25.638894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:25.639689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:25.641552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1031/2028] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:25 2022] DELETE /identity/v3/projects/20db06d2caf84ad2957fbdf0acb7deb0 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:25.643162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 998/2029] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:25.647456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:25.651347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:25.656006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf05dcc-29d1-4395-bd20-3eafc36b07a5 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:25.658468 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-dcf05dcc-29d1-4395-bd20-3eafc36b07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:25.741264 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf05dcc-29d1-4395-bd20-3eafc36b07a5 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:25.746230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530eb522-6899-491f-bd5e-5bd110792ed3 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:25.747107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f3c67-daaa-4ff1-af54-7988d5aa9d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:25.747797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3c67-daaa-4ff1-af54-7988d5aa9d6a service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:25.748139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3c67-daaa-4ff1-af54-7988d5aa9d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:25.748549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3c67-daaa-4ff1-af54-7988d5aa9d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:25.752840 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:25.753747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:25.754186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:25.754618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] PATH_INFO: `/v3/users/4543a34b51bb485eba438ae7d69ac2cd` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:25.756176 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4543a34b51bb485eba438ae7d69ac2cd)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:25.758958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:25.771152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f3c67-daaa-4ff1-af54-7988d5aa9d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:25.773160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f3c67-daaa-4ff1-af54-7988d5aa9d6a service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:25.808295 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 999/2030] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:51:26.169112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4543a34b51bb485eba438ae7d69ac2cd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:26.170738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:26.208539 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4543a34b51bb485eba438ae7d69ac2cd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:26.209684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4543a34b51bb485eba438ae7d69ac2cd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:26.210560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4543a34b51bb485eba438ae7d69ac2cd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:26.416284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4543a34b51bb485eba438ae7d69ac2cd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:26.439689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1032/2031] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:25 2022] DELETE /identity/v3/users/4543a34b51bb485eba438ae7d69ac2cd => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:26.457298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:26.469417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f3c67-daaa-4ff1-af54-7988d5aa9d6a service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:26.471445 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-bb6f3c67-daaa-4ff1-af54-7988d5aa9d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:26.485892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f3c67-daaa-4ff1-af54-7988d5aa9d6a service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:26.555539 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:26.556723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:26.557294 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:26.557824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] PATH_INFO: `/v3/projects/016b0c382e92427cbfe7a99c1104caa1` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:26.559775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=016b0c382e92427cbfe7a99c1104caa1)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:26.563482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:26.816319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016b0c382e92427cbfe7a99c1104caa1', 'request_id': 'req-8e2be006-f63a-4449-b997-9e3cff715957'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:26.826526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016b0c382e92427cbfe7a99c1104caa1', 'request_id': 'req-8e2be006-f63a-4449-b997-9e3cff715957'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:26.834538 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016b0c382e92427cbfe7a99c1104caa1', 'request_id': 'req-8e2be006-f63a-4449-b997-9e3cff715957'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.013634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] The token cache is being invalidate because project 016b0c382e92427cbfe7a99c1104caa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:27.014335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.016035 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8e2be006-f63a-4449-b997-9e3cff715957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.017591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1000/2032] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:26 2022] DELETE /identity/v3/projects/016b0c382e92427cbfe7a99c1104caa1 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:27.031733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:27.031733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:27.031930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2425a-d9db-491d-96b9-93b8853e5db1 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.031930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2425a-d9db-491d-96b9-93b8853e5db1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.032334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2425a-d9db-491d-96b9-93b8853e5db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.043269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d816f7d-ccf9-4831-a67a-a69982d3369b admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:27.055382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-95e2425a-d9db-491d-96b9-93b8853e5db1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:27.061502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-95e2425a-d9db-491d-96b9-93b8853e5db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:27.110484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:27.111122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.111405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.111679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] PATH_INFO: `/v3/projects/d2688a801e35458db8753304b0929f47` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.118306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2688a801e35458db8753304b0929f47)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:27.118306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:27.119426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1001/2033] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:27.127701 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b9dc1-a456-4909-ad7c-d34057dfa235 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.128033 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b9dc1-a456-4909-ad7c-d34057dfa235 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.128321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b9dc1-a456-4909-ad7c-d34057dfa235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.147607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-5e5b9dc1-a456-4909-ad7c-d34057dfa235 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:27.148868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5e5b9dc1-a456-4909-ad7c-d34057dfa235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:27.157985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1002/2034] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:27.162548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b9dc1-a456-4909-ad7c-d34057dfa235 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:27.172374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b9dc1-a456-4909-ad7c-d34057dfa235 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:27.177071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f6325-110b-4ede-95c0-498c5bce28f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:27.177673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f6325-110b-4ede-95c0-498c5bce28f2 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.177961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f6325-110b-4ede-95c0-498c5bce28f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.178243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f6325-110b-4ede-95c0-498c5bce28f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.178896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f6325-110b-4ede-95c0-498c5bce28f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:27.179203 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f6325-110b-4ede-95c0-498c5bce28f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:27.180838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f6325-110b-4ede-95c0-498c5bce28f2 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:27.192837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1003/2035] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:51:27.197075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f6325-110b-4ede-95c0-498c5bce28f2 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:27.206784 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f6325-110b-4ede-95c0-498c5bce28f2 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:27.211481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd0544-dad8-436c-b912-f2084dfaa2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:27.212119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd0544-dad8-436c-b912-f2084dfaa2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.212406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd0544-dad8-436c-b912-f2084dfaa2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.212684 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd0544-dad8-436c-b912-f2084dfaa2e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.215977 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd0544-dad8-436c-b912-f2084dfaa2e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:27.215977 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd0544-dad8-436c-b912-f2084dfaa2e9 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:27.269670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2688a801e35458db8753304b0929f47', 'request_id': 'req-d868d0ed-b5d0-4323-af00-38d7556c7f14'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.350764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1004/2036] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:27 2022] POST /identity/v3/projects => generated 397 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:51:27.350764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd0544-dad8-436c-b912-f2084dfaa2e9 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:27.352293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4fdd0544-dad8-436c-b912-f2084dfaa2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:27.387464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2688a801e35458db8753304b0929f47', 'request_id': 'req-d868d0ed-b5d0-4323-af00-38d7556c7f14'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.398894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2688a801e35458db8753304b0929f47', 'request_id': 'req-d868d0ed-b5d0-4323-af00-38d7556c7f14'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.416993 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] The token cache is being invalidate because project d2688a801e35458db8753304b0929f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:27.417912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.418482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.427282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1033/2037] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:27 2022] DELETE /identity/v3/projects/d2688a801e35458db8753304b0929f47 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:27.427282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:27.427282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:27.433520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd0544-dad8-436c-b912-f2084dfaa2e9 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:27.438767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:27.439412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.439691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.439996 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] PATH_INFO: `/v3/users/8664cd686f2e4261841db30a2fa88eac` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.441123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8664cd686f2e4261841db30a2fa88eac)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:27.458240 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:27.508224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868d0ed-b5d0-4323-af00-38d7556c7f14 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:27.513395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a932bf4-6058-4936-abb0-38d51c0a3161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:27.514019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a932bf4-6058-4936-abb0-38d51c0a3161 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.514306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a932bf4-6058-4936-abb0-38d51c0a3161 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.514595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a932bf4-6058-4936-abb0-38d51c0a3161 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.516207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a932bf4-6058-4936-abb0-38d51c0a3161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:27.518162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a932bf4-6058-4936-abb0-38d51c0a3161 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:27.538931 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:27.632369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:27.639122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1034/2038] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:27 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:27.641337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fcf1a-8ff1-40a0-b5a7-dd9e8f559356 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.641733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fcf1a-8ff1-40a0-b5a7-dd9e8f559356 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.642446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fcf1a-8ff1-40a0-b5a7-dd9e8f559356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.664704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-7a9fcf1a-8ff1-40a0-b5a7-dd9e8f559356 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:27.665369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-7a9fcf1a-8ff1-40a0-b5a7-dd9e8f559356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:27.673463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1035/2039] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:27.674271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fcf1a-8ff1-40a0-b5a7-dd9e8f559356 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:27.684178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fcf1a-8ff1-40a0-b5a7-dd9e8f559356 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:27.690007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2153f-9144-4edc-92e7-0585190e529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:27.693299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2153f-9144-4edc-92e7-0585190e529c admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.693299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2153f-9144-4edc-92e7-0585190e529c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.693299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2153f-9144-4edc-92e7-0585190e529c admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.693299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2153f-9144-4edc-92e7-0585190e529c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:27.693651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2153f-9144-4edc-92e7-0585190e529c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:27.704622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1036/2040] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:27.705479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8664cd686f2e4261841db30a2fa88eac'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.705954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556aabe-03d5-4758-ad61-6cd489270995 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.706245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556aabe-03d5-4758-ad61-6cd489270995 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.706535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556aabe-03d5-4758-ad61-6cd489270995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.706673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8664cd686f2e4261841db30a2fa88eac'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.719980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8664cd686f2e4261841db30a2fa88eac'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.734170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3556aabe-03d5-4758-ad61-6cd489270995 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:27.735386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3556aabe-03d5-4758-ad61-6cd489270995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:27.743293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1037/2041] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:27.744086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556aabe-03d5-4758-ad61-6cd489270995 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:27.753999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556aabe-03d5-4758-ad61-6cd489270995 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:27.758631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582844a-3c13-4ca5-9f9e-6b357eae4a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:27.764137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582844a-3c13-4ca5-9f9e-6b357eae4a40 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.764137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582844a-3c13-4ca5-9f9e-6b357eae4a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.764137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582844a-3c13-4ca5-9f9e-6b357eae4a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.764137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582844a-3c13-4ca5-9f9e-6b357eae4a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:27.764137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582844a-3c13-4ca5-9f9e-6b357eae4a40 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:27.774283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1038/2042] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:27.775351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582844a-3c13-4ca5-9f9e-6b357eae4a40 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:27.785916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582844a-3c13-4ca5-9f9e-6b357eae4a40 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:27.790583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d2ea3-f2dd-43f1-9173-29d77cfa31e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:27.791173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d2ea3-f2dd-43f1-9173-29d77cfa31e4 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.791456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d2ea3-f2dd-43f1-9173-29d77cfa31e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.791731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d2ea3-f2dd-43f1-9173-29d77cfa31e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.792417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d2ea3-f2dd-43f1-9173-29d77cfa31e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:27.792721 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d2ea3-f2dd-43f1-9173-29d77cfa31e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:27.794306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d2ea3-f2dd-43f1-9173-29d77cfa31e4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:27.806328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1039/2043] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:51:27.807680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d2ea3-f2dd-43f1-9173-29d77cfa31e4 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:27.818513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d2ea3-f2dd-43f1-9173-29d77cfa31e4 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:27.823046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d82790-19e5-4b13-9673-c98638a007b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:27.823680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d82790-19e5-4b13-9673-c98638a007b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.823987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d82790-19e5-4b13-9673-c98638a007b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.824265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d82790-19e5-4b13-9673-c98638a007b5 admin admin] PATH_INFO: `/v3/projects/d6d3b23f9762469ea091237823aebc4b/users/bcf2bbaf656147518cf40573f5f4e64c/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.839572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d82790-19e5-4b13-9673-c98638a007b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6d3b23f9762469ea091237823aebc4b, user_id=bcf2bbaf656147518cf40573f5f4e64c, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:27.841191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8664cd686f2e4261841db30a2fa88eac'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.842544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d82790-19e5-4b13-9673-c98638a007b5 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:27.849094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8664cd686f2e4261841db30a2fa88eac'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:27.855340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1005/2044] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:27 2022] DELETE /identity/v3/users/8664cd686f2e4261841db30a2fa88eac => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:27.856266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:27.857579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:27.923153 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eff001-2015-4b0c-acc9-6ea93851e12d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:27.924579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1040/2045] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:27 2022] PUT /identity/v3/projects/d6d3b23f9762469ea091237823aebc4b/users/bcf2bbaf656147518cf40573f5f4e64c/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:27.933406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d82790-19e5-4b13-9673-c98638a007b5 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:27.934778 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-38d82790-19e5-4b13-9673-c98638a007b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:27.988837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:27.989401 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:27.989690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:27.989976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] PATH_INFO: `/v3/users/5708a323e4ef443a9efabdbcc0425316` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:27.991127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5708a323e4ef443a9efabdbcc0425316)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:27.993087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:28.000284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d82790-19e5-4b13-9673-c98638a007b5 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:28.004989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d31e1f-4cef-448e-8ffa-8c095dfcf4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:28.005613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d31e1f-4cef-448e-8ffa-8c095dfcf4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.005949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d31e1f-4cef-448e-8ffa-8c095dfcf4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.006473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d31e1f-4cef-448e-8ffa-8c095dfcf4f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.007309 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d31e1f-4cef-448e-8ffa-8c095dfcf4f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:28.008783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:28.010952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d31e1f-4cef-448e-8ffa-8c095dfcf4f6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:28.136528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:28.146534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1041/2046] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:27 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:28.146534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024098ad-ff57-4028-9809-2abfe05e5137 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.146534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024098ad-ff57-4028-9809-2abfe05e5137 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.146534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024098ad-ff57-4028-9809-2abfe05e5137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.165962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-024098ad-ff57-4028-9809-2abfe05e5137 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:28.167217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-024098ad-ff57-4028-9809-2abfe05e5137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.228297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1042/2047] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:28.229742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e910c-82f7-4f3f-94cf-e372c5ffb3f3 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.230045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e910c-82f7-4f3f-94cf-e372c5ffb3f3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.230302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e910c-82f7-4f3f-94cf-e372c5ffb3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.258853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5708a323e4ef443a9efabdbcc0425316'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:28.259477 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5708a323e4ef443a9efabdbcc0425316'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:28.259610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-2d4e910c-82f7-4f3f-94cf-e372c5ffb3f3 None None] MFA Rules not processed for user `bcf2bbaf656147518cf40573f5f4e64c`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:28.260733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2d4e910c-82f7-4f3f-94cf-e372c5ffb3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.272319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5708a323e4ef443a9efabdbcc0425316'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:28.325460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1043/2048] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:28.326333 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e910c-82f7-4f3f-94cf-e372c5ffb3f3 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:28.338916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e910c-82f7-4f3f-94cf-e372c5ffb3f3 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:28.343684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73f129-1ae3-44a7-b039-f08eee2e0eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:28.344360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73f129-1ae3-44a7-b039-f08eee2e0eac admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.344652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73f129-1ae3-44a7-b039-f08eee2e0eac admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.344933 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73f129-1ae3-44a7-b039-f08eee2e0eac admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.345612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73f129-1ae3-44a7-b039-f08eee2e0eac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:28.347544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73f129-1ae3-44a7-b039-f08eee2e0eac admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:28.376364 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5708a323e4ef443a9efabdbcc0425316'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:28.383980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5708a323e4ef443a9efabdbcc0425316'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:28.390012 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1006/2049] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:27 2022] DELETE /identity/v3/users/5708a323e4ef443a9efabdbcc0425316 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:28.390885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:28.392323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.445770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1044/2050] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:28 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:28.445770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73f129-1ae3-44a7-b039-f08eee2e0eac admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:28.447526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-fd73f129-1ae3-44a7-b039-f08eee2e0eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.450514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76104fc8-a78e-46f8-8c7a-5bb928a6c031 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:28.454625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20247aea-3a27-4e7f-946d-e710a05eb7ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:28.455215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247aea-3a27-4e7f-946d-e710a05eb7ca service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.455517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247aea-3a27-4e7f-946d-e710a05eb7ca service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.455798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247aea-3a27-4e7f-946d-e710a05eb7ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.457706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-20247aea-3a27-4e7f-946d-e710a05eb7ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.506520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73f129-1ae3-44a7-b039-f08eee2e0eac admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:28.523617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20247aea-3a27-4e7f-946d-e710a05eb7ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:28.525000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20247aea-3a27-4e7f-946d-e710a05eb7ca service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:28.535935 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1007/2051] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:28.537496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ee95-2e0a-4055-8d22-5c07ad9a5aa9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.537712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ee95-2e0a-4055-8d22-5c07ad9a5aa9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.538110 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ee95-2e0a-4055-8d22-5c07ad9a5aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.559351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-61f4ee95-2e0a-4055-8d22-5c07ad9a5aa9 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:28.560984 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-61f4ee95-2e0a-4055-8d22-5c07ad9a5aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.568973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787f34f-ae57-4b25-aaa8-3d9a4eb4124b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:28.569686 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787f34f-ae57-4b25-aaa8-3d9a4eb4124b service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.569977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787f34f-ae57-4b25-aaa8-3d9a4eb4124b service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.570261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787f34f-ae57-4b25-aaa8-3d9a4eb4124b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.571281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1008/2052] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:28.572268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4ee95-2e0a-4055-8d22-5c07ad9a5aa9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:28.573650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-61f4ee95-2e0a-4055-8d22-5c07ad9a5aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.578764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c787f34f-ae57-4b25-aaa8-3d9a4eb4124b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.595535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4ee95-2e0a-4055-8d22-5c07ad9a5aa9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:28.601079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd4240-506b-406f-ad68-8842922ba1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:28.601715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd4240-506b-406f-ad68-8842922ba1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.602004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd4240-506b-406f-ad68-8842922ba1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.602284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd4240-506b-406f-ad68-8842922ba1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.602943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd4240-506b-406f-ad68-8842922ba1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:28.604619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd4240-506b-406f-ad68-8842922ba1a9 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:28.617570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1009/2053] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:28.617570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd4240-506b-406f-ad68-8842922ba1a9 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:28.624486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-51fd4240-506b-406f-ad68-8842922ba1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.634217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd4240-506b-406f-ad68-8842922ba1a9 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:28.638628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e2d1f-9f6e-4bf6-9772-e76395d1168a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:28.639432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e2d1f-9f6e-4bf6-9772-e76395d1168a service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.639752 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e2d1f-9f6e-4bf6-9772-e76395d1168a service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.640408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e2d1f-9f6e-4bf6-9772-e76395d1168a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.651207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e2d1f-9f6e-4bf6-9772-e76395d1168a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:28.653381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e2d1f-9f6e-4bf6-9772-e76395d1168a service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:28.662810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787f34f-ae57-4b25-aaa8-3d9a4eb4124b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:28.664403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787f34f-ae57-4b25-aaa8-3d9a4eb4124b service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:28.665188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1010/2054] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:28.665863 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e2d1f-9f6e-4bf6-9772-e76395d1168a service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:28.675572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1045/2055] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:28 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:28.676348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e2d1f-9f6e-4bf6-9772-e76395d1168a service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:28.683261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c34e57-1da9-445c-8a5b-f45b9b80de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:28.684059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c34e57-1da9-445c-8a5b-f45b9b80de0b admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.684154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c34e57-1da9-445c-8a5b-f45b9b80de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.684440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c34e57-1da9-445c-8a5b-f45b9b80de0b admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.686360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c34e57-1da9-445c-8a5b-f45b9b80de0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:28.687719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c34e57-1da9-445c-8a5b-f45b9b80de0b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:28.700670 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1011/2056] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:28.704649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787f34f-ae57-4b25-aaa8-3d9a4eb4124b service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:28.721677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787f34f-ae57-4b25-aaa8-3d9a4eb4124b service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:28.727183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fa8d1-949c-4fbe-8af7-76bd1ec44809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:28.728218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fa8d1-949c-4fbe-8af7-76bd1ec44809 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.728607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fa8d1-949c-4fbe-8af7-76bd1ec44809 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.728989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fa8d1-949c-4fbe-8af7-76bd1ec44809 admin admin] PATH_INFO: `/v3/projects/9bf0cb62f7794f328d483c2ea1f054e3/users/cba52c999e414b96b5183cbc7ff9ea0f/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.747433 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fa8d1-949c-4fbe-8af7-76bd1ec44809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf0cb62f7794f328d483c2ea1f054e3, user_id=cba52c999e414b96b5183cbc7ff9ea0f, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:28.750714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fa8d1-949c-4fbe-8af7-76bd1ec44809 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:28.803589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:28.856368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1046/2057] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:28 2022] PUT /identity/v3/projects/9bf0cb62f7794f328d483c2ea1f054e3/users/cba52c999e414b96b5183cbc7ff9ea0f/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:28.866738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a748fc-e90c-40ec-89a7-75319e896cb5 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.867292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a748fc-e90c-40ec-89a7-75319e896cb5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.867382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a748fc-e90c-40ec-89a7-75319e896cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.909210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c8a748fc-e90c-40ec-89a7-75319e896cb5 None None] MFA Rules not processed for user `cba52c999e414b96b5183cbc7ff9ea0f`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:28.910614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c8a748fc-e90c-40ec-89a7-75319e896cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.962669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:28.996593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1012/2058] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:29.002846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fa8d1-949c-4fbe-8af7-76bd1ec44809 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:29.071271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fa8d1-949c-4fbe-8af7-76bd1ec44809 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:29.076164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a801e9a-dee5-4e62-9b04-e1ce2336c0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:29.076970 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a801e9a-dee5-4e62-9b04-e1ce2336c0cf admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:29.077259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a801e9a-dee5-4e62-9b04-e1ce2336c0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:29.078039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a801e9a-dee5-4e62-9b04-e1ce2336c0cf admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:29.078699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a801e9a-dee5-4e62-9b04-e1ce2336c0cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:29.080659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a801e9a-dee5-4e62-9b04-e1ce2336c0cf admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:29.179619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a748fc-e90c-40ec-89a7-75319e896cb5 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:29.184781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c8a748fc-e90c-40ec-89a7-75319e896cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:29.196803 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a748fc-e90c-40ec-89a7-75319e896cb5 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:29.203829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:29.204524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:29.204812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:29.205095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] PATH_INFO: `/v3/projects/d1585277ba5040a38159a4c072568c80` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:29.206285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1585277ba5040a38159a4c072568c80)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:29.208282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:29.236128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1047/2059] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:29 2022] POST /identity/v3/projects => generated 377 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:29.244383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a801e9a-dee5-4e62-9b04-e1ce2336c0cf admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:29.326167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a801e9a-dee5-4e62-9b04-e1ce2336c0cf admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:29.333119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356a762-e508-4e56-b7d1-97c0befbab0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:29.333806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356a762-e508-4e56-b7d1-97c0befbab0b admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:29.335850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356a762-e508-4e56-b7d1-97c0befbab0b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:29.335850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356a762-e508-4e56-b7d1-97c0befbab0b admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:29.335850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9356a762-e508-4e56-b7d1-97c0befbab0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:29.337616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9356a762-e508-4e56-b7d1-97c0befbab0b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:29.389284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1585277ba5040a38159a4c072568c80', 'request_id': 'req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:29.398571 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1585277ba5040a38159a4c072568c80', 'request_id': 'req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:29.407411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1585277ba5040a38159a4c072568c80', 'request_id': 'req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:29.424310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1048/2060] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:29 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:29.426909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356a762-e508-4e56-b7d1-97c0befbab0b admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:29.438857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356a762-e508-4e56-b7d1-97c0befbab0b admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:29.510641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073232d2-99db-40ee-97d0-d6322d5bd642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:29.511475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073232d2-99db-40ee-97d0-d6322d5bd642 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:29.511596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073232d2-99db-40ee-97d0-d6322d5bd642 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:29.511828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073232d2-99db-40ee-97d0-d6322d5bd642 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:29.512523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073232d2-99db-40ee-97d0-d6322d5bd642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:29.514549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073232d2-99db-40ee-97d0-d6322d5bd642 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:29.518735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] The token cache is being invalidate because project d1585277ba5040a38159a4c072568c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:29.522059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:29.522059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:29.525664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1013/2061] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:29 2022] DELETE /identity/v3/projects/d1585277ba5040a38159a4c072568c80 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:29.526833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:29.529312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:29.533143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1049/2062] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:51:29.534365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073232d2-99db-40ee-97d0-d6322d5bd642 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:29.538889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-073232d2-99db-40ee-97d0-d6322d5bd642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:29.549802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55509a-5a5b-475b-a9b0-a9e8c3e72d7c admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:29.555425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073232d2-99db-40ee-97d0-d6322d5bd642 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:29.555944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:29.556050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:29.556263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:29.556545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] PATH_INFO: `/v3/users/661af264b9114d40a4c4e64915bc0dcc` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:29.558229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] RBAC: Authorizing `identity:delete_user(user_id=661af264b9114d40a4c4e64915bc0dcc)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:29.559999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:29.561177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:29.561543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:29.562561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] PATH_INFO: `/v3/projects/c0bdeb172d0145ba9804c7c10b54bdf0` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:29.563922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0bdeb172d0145ba9804c7c10b54bdf0)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:29.566117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:29.571671 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:29.876360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661af264b9114d40a4c4e64915bc0dcc'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:29.885632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661af264b9114d40a4c4e64915bc0dcc'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:29.894305 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661af264b9114d40a4c4e64915bc0dcc'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:29.894622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0bdeb172d0145ba9804c7c10b54bdf0', 'request_id': 'req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:29.963598 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661af264b9114d40a4c4e64915bc0dcc'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:29.971750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661af264b9114d40a4c4e64915bc0dcc'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:29.978007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1014/2063] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:29 2022] DELETE /identity/v3/users/661af264b9114d40a4c4e64915bc0dcc => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:29.978905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:29.980268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:30.048103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0bdeb172d0145ba9804c7c10b54bdf0', 'request_id': 'req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.054195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b42a5-8f7e-4c6c-8b36-2a91b5ecf19f admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:30.059597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0bdeb172d0145ba9804c7c10b54bdf0', 'request_id': 'req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.065453 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18a094-7049-40e2-b76f-9c8a75316d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:30.066400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a094-7049-40e2-b76f-9c8a75316d50 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:30.066882 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a094-7049-40e2-b76f-9c8a75316d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:30.067909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a094-7049-40e2-b76f-9c8a75316d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:30.069204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18a094-7049-40e2-b76f-9c8a75316d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:30.071413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18a094-7049-40e2-b76f-9c8a75316d50 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:30.077557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] The token cache is being invalidate because project c0bdeb172d0145ba9804c7c10b54bdf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:30.078013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.079021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.084239 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1050/2064] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:29 2022] DELETE /identity/v3/projects/c0bdeb172d0145ba9804c7c10b54bdf0 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:30.086053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1015/2065] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:30.089771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:30.090367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18a094-7049-40e2-b76f-9c8a75316d50 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:30.091498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:30.092762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0e18a094-7049-40e2-b76f-9c8a75316d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:30.106382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18a094-7049-40e2-b76f-9c8a75316d50 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:30.111957 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8e2a0-ab46-4a10-9a0a-9e2b38c57b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:30.112665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8e2a0-ab46-4a10-9a0a-9e2b38c57b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:30.112953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8e2a0-ab46-4a10-9a0a-9e2b38c57b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:30.113237 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8e2a0-ab46-4a10-9a0a-9e2b38c57b62 admin admin] PATH_INFO: `/v3/projects/ced93dd26ce64cf2a635078ba6341ccb/users/d5ee9970e6454e65a427932756786fb5/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:30.128638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8e2a0-ab46-4a10-9a0a-9e2b38c57b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ced93dd26ce64cf2a635078ba6341ccb, user_id=d5ee9970e6454e65a427932756786fb5, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:30.131740 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8e2a0-ab46-4a10-9a0a-9e2b38c57b62 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:30.160971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58a6f7-fe28-405a-a87f-3ee5e252ae5b admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:30.166126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:30.166781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:30.167062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:30.167339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] PATH_INFO: `/v3/users/dd5281d87fda40719df40414a316afd5` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:30.168544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd5281d87fda40719df40414a316afd5)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:30.170683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:30.242494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1016/2066] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:30 2022] PUT /identity/v3/projects/ced93dd26ce64cf2a635078ba6341ccb/users/d5ee9970e6454e65a427932756786fb5/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:30.258293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64588c4c-56b8-41f7-b3b2-c4dab933bb88 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:30.258595 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64588c4c-56b8-41f7-b3b2-c4dab933bb88 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:30.258875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64588c4c-56b8-41f7-b3b2-c4dab933bb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:30.309016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-64588c4c-56b8-41f7-b3b2-c4dab933bb88 None None] MFA Rules not processed for user `d5ee9970e6454e65a427932756786fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:30.311817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-64588c4c-56b8-41f7-b3b2-c4dab933bb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:30.384553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1017/2067] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:30.388814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64588c4c-56b8-41f7-b3b2-c4dab933bb88 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:30.409900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd5281d87fda40719df40414a316afd5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.427018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd5281d87fda40719df40414a316afd5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.427896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd5281d87fda40719df40414a316afd5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.428631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd5281d87fda40719df40414a316afd5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.463269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64588c4c-56b8-41f7-b3b2-c4dab933bb88 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:30.468960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e24267-cd91-4c52-8fc5-a2dae642ae4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:30.472337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e24267-cd91-4c52-8fc5-a2dae642ae4f admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:30.472337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e24267-cd91-4c52-8fc5-a2dae642ae4f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:30.472337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e24267-cd91-4c52-8fc5-a2dae642ae4f admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:30.472337 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e24267-cd91-4c52-8fc5-a2dae642ae4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:30.473782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e24267-cd91-4c52-8fc5-a2dae642ae4f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:30.541575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd5281d87fda40719df40414a316afd5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.557098 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1051/2068] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:30 2022] DELETE /identity/v3/users/dd5281d87fda40719df40414a316afd5 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:30.558604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:30.580361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:30.596370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1018/2069] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:30 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:30.600062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e24267-cd91-4c52-8fc5-a2dae642ae4f admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:30.606332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-30e24267-cd91-4c52-8fc5-a2dae642ae4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:30.696405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66a786-0d5a-4007-81e5-449d2481987c admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:30.704644 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e24267-cd91-4c52-8fc5-a2dae642ae4f admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:30.705511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:30.706299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:30.706659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:30.707123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] PATH_INFO: `/v3/projects/a99c94f8863d451abd323fef71b4d801` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:30.708542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a99c94f8863d451abd323fef71b4d801)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:30.710735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:30.711904 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b24b5-7072-495d-9247-23f3bb8e5ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:30.712569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b24b5-7072-495d-9247-23f3bb8e5ffa admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:30.712857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b24b5-7072-495d-9247-23f3bb8e5ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:30.713140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b24b5-7072-495d-9247-23f3bb8e5ffa admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:30.713794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b24b5-7072-495d-9247-23f3bb8e5ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:30.716108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b24b5-7072-495d-9247-23f3bb8e5ffa admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:30.756659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:30.861448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1019/2070] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:30 2022] POST /identity/v3/users => generated 295 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:30.865614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b24b5-7072-495d-9247-23f3bb8e5ffa admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:30.877372 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b24b5-7072-495d-9247-23f3bb8e5ffa admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:30.878034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a99c94f8863d451abd323fef71b4d801', 'request_id': 'req-a93e859e-682b-467d-939a-446f3d52dee6'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.939753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5d96f-1341-4c4e-af68-1ad8cd7e4b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:30.940410 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5d96f-1341-4c4e-af68-1ad8cd7e4b80 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:30.940696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5d96f-1341-4c4e-af68-1ad8cd7e4b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:30.941287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5d96f-1341-4c4e-af68-1ad8cd7e4b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:30.941633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5d96f-1341-4c4e-af68-1ad8cd7e4b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:30.943418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5d96f-1341-4c4e-af68-1ad8cd7e4b80 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:30.959561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1020/2071] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:51:30.964442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5d96f-1341-4c4e-af68-1ad8cd7e4b80 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:30.976403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a99c94f8863d451abd323fef71b4d801', 'request_id': 'req-a93e859e-682b-467d-939a-446f3d52dee6'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.977164 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5d96f-1341-4c4e-af68-1ad8cd7e4b80 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:30.978048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:30.983696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0ca6f-d46b-46ad-973b-78b539fd7b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:30.986480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a99c94f8863d451abd323fef71b4d801', 'request_id': 'req-a93e859e-682b-467d-939a-446f3d52dee6'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.986592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0ca6f-d46b-46ad-973b-78b539fd7b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:30.986592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0ca6f-d46b-46ad-973b-78b539fd7b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:30.986592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0ca6f-d46b-46ad-973b-78b539fd7b00 admin admin] PATH_INFO: `/v3/projects/ea06c8e4c8414e90aa9dea9e2ca794b7/users/2a55d8db5bb44aa78f2340ff18ad1c7c/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:30.996589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] The token cache is being invalidate because project a99c94f8863d451abd323fef71b4d801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:30.996925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:30.997581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a93e859e-682b-467d-939a-446f3d52dee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:31.003203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1052/2072] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:30 2022] DELETE /identity/v3/projects/a99c94f8863d451abd323fef71b4d801 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:31.004914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17885bb-294d-4a20-b1e5-55dbc2ccbe5e None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.005035 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0ca6f-d46b-46ad-973b-78b539fd7b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea06c8e4c8414e90aa9dea9e2ca794b7, user_id=2a55d8db5bb44aa78f2340ff18ad1c7c, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.005504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17885bb-294d-4a20-b1e5-55dbc2ccbe5e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.005789 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17885bb-294d-4a20-b1e5-55dbc2ccbe5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.008405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0ca6f-d46b-46ad-973b-78b539fd7b00 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.026432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e17885bb-294d-4a20-b1e5-55dbc2ccbe5e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:31.027743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e17885bb-294d-4a20-b1e5-55dbc2ccbe5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:31.037471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1053/2073] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:31.046977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709362c8-fe1d-4b75-85f3-1b9093949c1c None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.047270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709362c8-fe1d-4b75-85f3-1b9093949c1c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.047551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709362c8-fe1d-4b75-85f3-1b9093949c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.067927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-709362c8-fe1d-4b75-85f3-1b9093949c1c None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:31.069161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-709362c8-fe1d-4b75-85f3-1b9093949c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:31.077137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1054/2074] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:31.081251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709362c8-fe1d-4b75-85f3-1b9093949c1c None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.091251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709362c8-fe1d-4b75-85f3-1b9093949c1c None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.096030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d37a6b-6da8-4b7b-9545-2a9eac3df933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.096633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d37a6b-6da8-4b7b-9545-2a9eac3df933 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.096929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d37a6b-6da8-4b7b-9545-2a9eac3df933 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.097306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d37a6b-6da8-4b7b-9545-2a9eac3df933 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.097959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d37a6b-6da8-4b7b-9545-2a9eac3df933 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:31.098147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d37a6b-6da8-4b7b-9545-2a9eac3df933 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.099943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d37a6b-6da8-4b7b-9545-2a9eac3df933 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.123695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1055/2075] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:51:31.128969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d37a6b-6da8-4b7b-9545-2a9eac3df933 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.139901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d37a6b-6da8-4b7b-9545-2a9eac3df933 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.147245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a244af2-eeb1-4d83-8136-2b27afe08f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.149832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a244af2-eeb1-4d83-8136-2b27afe08f27 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.149832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a244af2-eeb1-4d83-8136-2b27afe08f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.150456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a244af2-eeb1-4d83-8136-2b27afe08f27 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.151650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a244af2-eeb1-4d83-8136-2b27afe08f27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.156855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1021/2076] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:30 2022] PUT /identity/v3/projects/ea06c8e4c8414e90aa9dea9e2ca794b7/users/2a55d8db5bb44aa78f2340ff18ad1c7c/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:31.157046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a244af2-eeb1-4d83-8136-2b27afe08f27 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.158038 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0ca6f-d46b-46ad-973b-78b539fd7b00 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.162282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-35e0ca6f-d46b-46ad-973b-78b539fd7b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:31.261551 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0ca6f-d46b-46ad-973b-78b539fd7b00 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.266331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba8395-b39c-4011-8b3a-6e13039840fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.266945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8395-b39c-4011-8b3a-6e13039840fe admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.267231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8395-b39c-4011-8b3a-6e13039840fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.267514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8395-b39c-4011-8b3a-6e13039840fe admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.268200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba8395-b39c-4011-8b3a-6e13039840fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.269820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba8395-b39c-4011-8b3a-6e13039840fe admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.283014 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1022/2077] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:31.284054 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba8395-b39c-4011-8b3a-6e13039840fe admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.285914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-35ba8395-b39c-4011-8b3a-6e13039840fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:31.304542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba8395-b39c-4011-8b3a-6e13039840fe admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.312682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1056/2078] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:31 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:51:31.314518 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a244af2-eeb1-4d83-8136-2b27afe08f27 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.322822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.323942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.324432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.324873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] PATH_INFO: `/v3/projects/12ba31dd37c9447b82eb18c66c14f980` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.326971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12ba31dd37c9447b82eb18c66c14f980)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.331506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.404640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a244af2-eeb1-4d83-8136-2b27afe08f27 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.411798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54519001-c707-4210-8560-ec4dd634b5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.412505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54519001-c707-4210-8560-ec4dd634b5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.412833 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54519001-c707-4210-8560-ec4dd634b5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.413056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54519001-c707-4210-8560-ec4dd634b5a9 admin admin] PATH_INFO: `/v3/projects/ea06c8e4c8414e90aa9dea9e2ca794b7/users/2a55d8db5bb44aa78f2340ff18ad1c7c/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.415425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54519001-c707-4210-8560-ec4dd634b5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea06c8e4c8414e90aa9dea9e2ca794b7, user_id=2a55d8db5bb44aa78f2340ff18ad1c7c, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.418242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54519001-c707-4210-8560-ec4dd634b5a9 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.429818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1057/2079] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:31 2022] PUT /identity/v3/projects/ea06c8e4c8414e90aa9dea9e2ca794b7/users/2a55d8db5bb44aa78f2340ff18ad1c7c/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:31.431449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54519001-c707-4210-8560-ec4dd634b5a9 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.482099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12ba31dd37c9447b82eb18c66c14f980', 'request_id': 'req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:31.489059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12ba31dd37c9447b82eb18c66c14f980', 'request_id': 'req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:31.499548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12ba31dd37c9447b82eb18c66c14f980', 'request_id': 'req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:31.514202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54519001-c707-4210-8560-ec4dd634b5a9 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.576022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8818f-e3f6-47d7-9bb1-40540fe667b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.576651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8818f-e3f6-47d7-9bb1-40540fe667b6 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.576965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8818f-e3f6-47d7-9bb1-40540fe667b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.577293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8818f-e3f6-47d7-9bb1-40540fe667b6 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.577938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8818f-e3f6-47d7-9bb1-40540fe667b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.579908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8818f-e3f6-47d7-9bb1-40540fe667b6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.588085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] The token cache is being invalidate because project 12ba31dd37c9447b82eb18c66c14f980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:31.588383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:31.591137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aa30ddbe-867d-4f3a-99e2-f4f191d36e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:31.591137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1023/2080] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:31 2022] DELETE /identity/v3/projects/12ba31dd37c9447b82eb18c66c14f980 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:31.592465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35dc99-6ec5-404e-bb76-d7f73cd56850 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.593889 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35dc99-6ec5-404e-bb76-d7f73cd56850 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.594172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35dc99-6ec5-404e-bb76-d7f73cd56850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.623914 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ec35dc99-6ec5-404e-bb76-d7f73cd56850 None None] MFA Rules not processed for user `2a55d8db5bb44aa78f2340ff18ad1c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:31.625432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ec35dc99-6ec5-404e-bb76-d7f73cd56850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:31.700608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1024/2081] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:31.701220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1058/2082] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:31 2022] POST /identity/v3/users => generated 291 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:31.702242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320b8ab-9748-4c29-8c46-a8e9ac37b0de None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.702566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320b8ab-9748-4c29-8c46-a8e9ac37b0de None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.702911 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320b8ab-9748-4c29-8c46-a8e9ac37b0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.705385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8818f-e3f6-47d7-9bb1-40540fe667b6 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.708691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e8d8818f-e3f6-47d7-9bb1-40540fe667b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:31.723441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8818f-e3f6-47d7-9bb1-40540fe667b6 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.729616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc02df-9328-44eb-b9e7-9bb655993194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.730345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc02df-9328-44eb-b9e7-9bb655993194 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.730737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc02df-9328-44eb-b9e7-9bb655993194 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.731147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc02df-9328-44eb-b9e7-9bb655993194 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.731241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7320b8ab-9748-4c29-8c46-a8e9ac37b0de None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:31.732023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc02df-9328-44eb-b9e7-9bb655993194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.733057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7320b8ab-9748-4c29-8c46-a8e9ac37b0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:31.734740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc02df-9328-44eb-b9e7-9bb655993194 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.758033 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1025/2083] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:31.759186 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320b8ab-9748-4c29-8c46-a8e9ac37b0de None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.761602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7320b8ab-9748-4c29-8c46-a8e9ac37b0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:31.770060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1059/2084] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:31.771651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023e6c-149d-4218-a4a4-6063f7ac06a4 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.775157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023e6c-149d-4218-a4a4-6063f7ac06a4 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.775661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023e6c-149d-4218-a4a4-6063f7ac06a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.802017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-de023e6c-149d-4218-a4a4-6063f7ac06a4 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:31.803377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-de023e6c-149d-4218-a4a4-6063f7ac06a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:31.815485 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1060/2085] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:31.815485 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de023e6c-149d-4218-a4a4-6063f7ac06a4 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.826443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de023e6c-149d-4218-a4a4-6063f7ac06a4 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.831903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bd6d4-69de-44c3-812c-c7d38e7e4eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.832580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bd6d4-69de-44c3-812c-c7d38e7e4eef admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.832967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bd6d4-69de-44c3-812c-c7d38e7e4eef admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.833357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bd6d4-69de-44c3-812c-c7d38e7e4eef admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.834085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9bd6d4-69de-44c3-812c-c7d38e7e4eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.836034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320b8ab-9748-4c29-8c46-a8e9ac37b0de None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.836158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9bd6d4-69de-44c3-812c-c7d38e7e4eef admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.844593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4321ef5-c0e6-4c46-875b-59f49d0eb93a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.845220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4321ef5-c0e6-4c46-875b-59f49d0eb93a service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.845505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4321ef5-c0e6-4c46-875b-59f49d0eb93a service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.845789 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4321ef5-c0e6-4c46-875b-59f49d0eb93a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.847862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1061/2086] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:31.848898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bd6d4-69de-44c3-812c-c7d38e7e4eef admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.857041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4321ef5-c0e6-4c46-875b-59f49d0eb93a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.858979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4321ef5-c0e6-4c46-875b-59f49d0eb93a service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.861882 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bd6d4-69de-44c3-812c-c7d38e7e4eef admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.868080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa4629-de33-42ac-89f3-5c4be56bd00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.869006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa4629-de33-42ac-89f3-5c4be56bd00a admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.869868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa4629-de33-42ac-89f3-5c4be56bd00a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.870303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa4629-de33-42ac-89f3-5c4be56bd00a admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.870915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1026/2087] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:31.871972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4321ef5-c0e6-4c46-875b-59f49d0eb93a service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.873297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa4629-de33-42ac-89f3-5c4be56bd00a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:31.873606 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa4629-de33-42ac-89f3-5c4be56bd00a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.879551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa4629-de33-42ac-89f3-5c4be56bd00a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.891815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4321ef5-c0e6-4c46-875b-59f49d0eb93a service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.897662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7775bad-712f-48f3-a36a-6ac9ec3823ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.898328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7775bad-712f-48f3-a36a-6ac9ec3823ad admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.898616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7775bad-712f-48f3-a36a-6ac9ec3823ad admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.898894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7775bad-712f-48f3-a36a-6ac9ec3823ad admin admin] PATH_INFO: `/v3/projects/60ee4c70bf8344b793bb990629c253f3/users/5df2cf652d6f4f849de71502e8d03d02/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.904243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1062/2088] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:51:31.906171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa4629-de33-42ac-89f3-5c4be56bd00a admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:31.915202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7775bad-712f-48f3-a36a-6ac9ec3823ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=60ee4c70bf8344b793bb990629c253f3, user_id=5df2cf652d6f4f849de71502e8d03d02, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.916110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa4629-de33-42ac-89f3-5c4be56bd00a admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:31.917905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7775bad-712f-48f3-a36a-6ac9ec3823ad admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:31.920783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378d9b0-9c70-410e-8e60-72bd560741b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:31.921415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378d9b0-9c70-410e-8e60-72bd560741b0 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:31.921697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378d9b0-9c70-410e-8e60-72bd560741b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:31.921971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378d9b0-9c70-410e-8e60-72bd560741b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:31.922594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378d9b0-9c70-410e-8e60-72bd560741b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:31.924513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378d9b0-9c70-410e-8e60-72bd560741b0 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.016441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1027/2089] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:31 2022] PUT /identity/v3/projects/60ee4c70bf8344b793bb990629c253f3/users/5df2cf652d6f4f849de71502e8d03d02/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:32.017267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7775bad-712f-48f3-a36a-6ac9ec3823ad admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.018877 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b7775bad-712f-48f3-a36a-6ac9ec3823ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:32.046640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1063/2090] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:31 2022] POST /identity/v3/projects => generated 393 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:32.046640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e50c-bdc5-4164-bc2d-501c6b413918 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.046640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e50c-bdc5-4164-bc2d-501c6b413918 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.046640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e50c-bdc5-4164-bc2d-501c6b413918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.073238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3676e50c-bdc5-4164-bc2d-501c6b413918 None None] MFA Rules not processed for user `5df2cf652d6f4f849de71502e8d03d02`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:32.074543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3676e50c-bdc5-4164-bc2d-501c6b413918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:32.078642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7775bad-712f-48f3-a36a-6ac9ec3823ad admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.126850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4fe3-e674-41fa-b717-96171b855f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.127477 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4fe3-e674-41fa-b717-96171b855f88 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.127772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4fe3-e674-41fa-b717-96171b855f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.128091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4fe3-e674-41fa-b717-96171b855f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.129928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b8cf4fe3-e674-41fa-b717-96171b855f88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:32.141575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1064/2091] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:32.142406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676e50c-bdc5-4164-bc2d-501c6b413918 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.192044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf4fe3-e674-41fa-b717-96171b855f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.193489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf4fe3-e674-41fa-b717-96171b855f88 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.203336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1028/2092] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:32.204273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4fe3-e674-41fa-b717-96171b855f88 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.213147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676e50c-bdc5-4164-bc2d-501c6b413918 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.214331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0360e89-5622-46e4-9656-30258a516600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.214954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0360e89-5622-46e4-9656-30258a516600 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.215061 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4fe3-e674-41fa-b717-96171b855f88 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.215249 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0360e89-5622-46e4-9656-30258a516600 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.215531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0360e89-5622-46e4-9656-30258a516600 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.216213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0360e89-5622-46e4-9656-30258a516600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.218552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0360e89-5622-46e4-9656-30258a516600 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.220663 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fc274-389d-42af-8206-abd3ff683c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.221399 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fc274-389d-42af-8206-abd3ff683c0e admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.221693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fc274-389d-42af-8206-abd3ff683c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.221976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fc274-389d-42af-8206-abd3ff683c0e admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.222612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350fc274-389d-42af-8206-abd3ff683c0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.224653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350fc274-389d-42af-8206-abd3ff683c0e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.359234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1065/2093] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:32 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:32.360723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1029/2094] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:32 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:32.365271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0360e89-5622-46e4-9656-30258a516600 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.365412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fc274-389d-42af-8206-abd3ff683c0e admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.376057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0360e89-5622-46e4-9656-30258a516600 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.439145 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a8b73-4a46-4f2d-b175-0e4d67ddf227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.439852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a8b73-4a46-4f2d-b175-0e4d67ddf227 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.440241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a8b73-4a46-4f2d-b175-0e4d67ddf227 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.440527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a8b73-4a46-4f2d-b175-0e4d67ddf227 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.441325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a8b73-4a46-4f2d-b175-0e4d67ddf227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.443384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a8b73-4a46-4f2d-b175-0e4d67ddf227 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.445060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fc274-389d-42af-8206-abd3ff683c0e admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.454598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1066/2095] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:32.456377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88953ae3-b23d-45c8-bc11-81e6cf00ad10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.457116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88953ae3-b23d-45c8-bc11-81e6cf00ad10 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.457552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88953ae3-b23d-45c8-bc11-81e6cf00ad10 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.457894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88953ae3-b23d-45c8-bc11-81e6cf00ad10 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.458551 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88953ae3-b23d-45c8-bc11-81e6cf00ad10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.460576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88953ae3-b23d-45c8-bc11-81e6cf00ad10 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.461036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a8b73-4a46-4f2d-b175-0e4d67ddf227 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.474473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a8b73-4a46-4f2d-b175-0e4d67ddf227 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.479489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf6879-e77b-4dd0-9644-61a567f72bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.480141 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6879-e77b-4dd0-9644-61a567f72bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.480427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6879-e77b-4dd0-9644-61a567f72bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.480733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6879-e77b-4dd0-9644-61a567f72bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.481382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf6879-e77b-4dd0-9644-61a567f72bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.483090 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf6879-e77b-4dd0-9644-61a567f72bc3 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.495900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1067/2096] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:32.496504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:32.498540 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf6879-e77b-4dd0-9644-61a567f72bc3 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.510124 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf6879-e77b-4dd0-9644-61a567f72bc3 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.516856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80315dfd-27ee-4983-8579-11586b237959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.517564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80315dfd-27ee-4983-8579-11586b237959 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.517819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80315dfd-27ee-4983-8579-11586b237959 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.518096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80315dfd-27ee-4983-8579-11586b237959 admin admin] PATH_INFO: `/v3/projects/ffc0ffa8142e4165b7e9045e27fc7611/users/97326c4e4181471fbaefb16427b8e50e/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.533451 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80315dfd-27ee-4983-8579-11586b237959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc0ffa8142e4165b7e9045e27fc7611, user_id=97326c4e4181471fbaefb16427b8e50e, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.536141 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80315dfd-27ee-4983-8579-11586b237959 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.579158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1030/2097] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:32 2022] POST /identity/v3/users => generated 298 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:32.580273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88953ae3-b23d-45c8-bc11-81e6cf00ad10 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.590356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88953ae3-b23d-45c8-bc11-81e6cf00ad10 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.595205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effffb9-1588-4971-a36e-336b8f193a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.595842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effffb9-1588-4971-a36e-336b8f193a43 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.596166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effffb9-1588-4971-a36e-336b8f193a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.596445 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effffb9-1588-4971-a36e-336b8f193a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.597107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effffb9-1588-4971-a36e-336b8f193a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.598711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effffb9-1588-4971-a36e-336b8f193a43 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.610449 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1031/2098] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:32.614676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effffb9-1588-4971-a36e-336b8f193a43 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.625540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effffb9-1588-4971-a36e-336b8f193a43 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.630270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db377b-88d9-4663-8221-a480ec254e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.630931 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db377b-88d9-4663-8221-a480ec254e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.631217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db377b-88d9-4663-8221-a480ec254e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.631498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db377b-88d9-4663-8221-a480ec254e9f admin admin] PATH_INFO: `/v3/projects/bd1e7104683a4b08a136a189cc97668c/users/ec8ba80dbc1c4edd8d6aa3242db3b07f/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.650705 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db377b-88d9-4663-8221-a480ec254e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1e7104683a4b08a136a189cc97668c, user_id=ec8ba80dbc1c4edd8d6aa3242db3b07f, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.650705 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db377b-88d9-4663-8221-a480ec254e9f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.679029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1068/2099] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:32 2022] PUT /identity/v3/projects/ffc0ffa8142e4165b7e9045e27fc7611/users/97326c4e4181471fbaefb16427b8e50e/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:32.690480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a92c95-9c71-4dc9-9c2d-271f89b34982 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.690770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a92c95-9c71-4dc9-9c2d-271f89b34982 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.691049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a92c95-9c71-4dc9-9c2d-271f89b34982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.719897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-46a92c95-9c71-4dc9-9c2d-271f89b34982 None None] MFA Rules not processed for user `97326c4e4181471fbaefb16427b8e50e`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:32.721125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-46a92c95-9c71-4dc9-9c2d-271f89b34982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:32.765765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1032/2100] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:32 2022] PUT /identity/v3/projects/bd1e7104683a4b08a136a189cc97668c/users/ec8ba80dbc1c4edd8d6aa3242db3b07f/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:32.769788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db377b-88d9-4663-8221-a480ec254e9f admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.788400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1069/2101] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:32.793318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a92c95-9c71-4dc9-9c2d-271f89b34982 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.846085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db377b-88d9-4663-8221-a480ec254e9f admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.851613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d2abe-9212-45f0-ad87-19132cfd65e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.852276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d2abe-9212-45f0-ad87-19132cfd65e9 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.852565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d2abe-9212-45f0-ad87-19132cfd65e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.852864 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d2abe-9212-45f0-ad87-19132cfd65e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.853672 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d2abe-9212-45f0-ad87-19132cfd65e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.855279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d2abe-9212-45f0-ad87-19132cfd65e9 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.866752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a92c95-9c71-4dc9-9c2d-271f89b34982 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.868219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1033/2102] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:32.871720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df779b40-98a1-4fca-a450-fee25090554c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.872457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df779b40-98a1-4fca-a450-fee25090554c admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.872747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df779b40-98a1-4fca-a450-fee25090554c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.873023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df779b40-98a1-4fca-a450-fee25090554c admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.873324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d2abe-9212-45f0-ad87-19132cfd65e9 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:32.873720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df779b40-98a1-4fca-a450-fee25090554c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.875626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df779b40-98a1-4fca-a450-fee25090554c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.888487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d2abe-9212-45f0-ad87-19132cfd65e9 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:32.892077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:32.899257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca38b1-9e58-4def-9749-4983878f027b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:32.899257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca38b1-9e58-4def-9749-4983878f027b admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.899257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca38b1-9e58-4def-9749-4983878f027b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.899257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca38b1-9e58-4def-9749-4983878f027b admin admin] PATH_INFO: `/v3/projects/bd1e7104683a4b08a136a189cc97668c/users/ec8ba80dbc1c4edd8d6aa3242db3b07f/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.899722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca38b1-9e58-4def-9749-4983878f027b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1e7104683a4b08a136a189cc97668c, user_id=ec8ba80dbc1c4edd8d6aa3242db3b07f, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:32.902265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca38b1-9e58-4def-9749-4983878f027b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:32.914063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1034/2103] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:32 2022] PUT /identity/v3/projects/bd1e7104683a4b08a136a189cc97668c/users/ec8ba80dbc1c4edd8d6aa3242db3b07f/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:32.924502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc8ba8-e9d4-4fdf-85e8-ff7373acb3f9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:32.924788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc8ba8-e9d4-4fdf-85e8-ff7373acb3f9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:32.925100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc8ba8-e9d4-4fdf-85e8-ff7373acb3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:32.957651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-27fc8ba8-e9d4-4fdf-85e8-ff7373acb3f9 None None] MFA Rules not processed for user `ec8ba80dbc1c4edd8d6aa3242db3b07f`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:32.958882 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-27fc8ba8-e9d4-4fdf-85e8-ff7373acb3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:32.996075 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1070/2104] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:32 2022] POST /identity/v3/projects => generated 393 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:51:33.000834 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df779b40-98a1-4fca-a450-fee25090554c admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:33.041974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1035/2105] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:33.054841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc8ba8-e9d4-4fdf-85e8-ff7373acb3f9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:33.076352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df779b40-98a1-4fca-a450-fee25090554c admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:33.082308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a89b-52a6-4560-9fcc-b50acc59c9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:33.082937 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a89b-52a6-4560-9fcc-b50acc59c9bf admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:33.083365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a89b-52a6-4560-9fcc-b50acc59c9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:33.083642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a89b-52a6-4560-9fcc-b50acc59c9bf admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:33.084305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348a89b-52a6-4560-9fcc-b50acc59c9bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:33.086242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348a89b-52a6-4560-9fcc-b50acc59c9bf admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:33.128890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc8ba8-e9d4-4fdf-85e8-ff7373acb3f9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:33.181057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3e755-d856-4b36-a0f7-2f373b678db7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:33.181707 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3e755-d856-4b36-a0f7-2f373b678db7 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:33.182001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3e755-d856-4b36-a0f7-2f373b678db7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:33.182280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3e755-d856-4b36-a0f7-2f373b678db7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:33.195230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1071/2106] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:32 2022] POST /identity/v3/users => generated 303 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:51:33.201611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a89b-52a6-4560-9fcc-b50acc59c9bf admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:33.214476 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a89b-52a6-4560-9fcc-b50acc59c9bf admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:33.221179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281a1af-a19b-47d0-a737-7136260580a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:33.222018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281a1af-a19b-47d0-a737-7136260580a7 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:33.222403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281a1af-a19b-47d0-a737-7136260580a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:33.222780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281a1af-a19b-47d0-a737-7136260580a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:33.223579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281a1af-a19b-47d0-a737-7136260580a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:33.225555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281a1af-a19b-47d0-a737-7136260580a7 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:33.237917 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1072/2107] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:33.242485 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281a1af-a19b-47d0-a737-7136260580a7 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:33.253772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281a1af-a19b-47d0-a737-7136260580a7 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:33.257026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3e755-d856-4b36-a0f7-2f373b678db7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:33.258514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3e755-d856-4b36-a0f7-2f373b678db7 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:33.260539 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89244805-b8af-48f9-902b-449ac3c265ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:33.260878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89244805-b8af-48f9-902b-449ac3c265ae admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:33.261172 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89244805-b8af-48f9-902b-449ac3c265ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:33.261495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89244805-b8af-48f9-902b-449ac3c265ae admin admin] PATH_INFO: `/v3/projects/d8b3a961e0cc4224988fbc599a9d66b6/users/00ac8f6353c841eaba1e152d927de90b/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:33.270535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1036/2108] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:51:33.279613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89244805-b8af-48f9-902b-449ac3c265ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b3a961e0cc4224988fbc599a9d66b6, user_id=00ac8f6353c841eaba1e152d927de90b, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:33.283421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89244805-b8af-48f9-902b-449ac3c265ae admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:33.379926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1073/2109] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:33 2022] PUT /identity/v3/projects/d8b3a961e0cc4224988fbc599a9d66b6/users/00ac8f6353c841eaba1e152d927de90b/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:33.384039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3e755-d856-4b36-a0f7-2f373b678db7 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:33.394860 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3e755-d856-4b36-a0f7-2f373b678db7 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:33.441457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89244805-b8af-48f9-902b-449ac3c265ae admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:33.479080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de583e26-9b8e-4527-b21d-4182f5e59167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:33.480154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de583e26-9b8e-4527-b21d-4182f5e59167 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:33.480871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de583e26-9b8e-4527-b21d-4182f5e59167 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:33.481312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de583e26-9b8e-4527-b21d-4182f5e59167 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:33.481990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de583e26-9b8e-4527-b21d-4182f5e59167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:33.484414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de583e26-9b8e-4527-b21d-4182f5e59167 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:33.495915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1037/2110] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:33.499114 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de583e26-9b8e-4527-b21d-4182f5e59167 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:33.510558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de583e26-9b8e-4527-b21d-4182f5e59167 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:33.514908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89244805-b8af-48f9-902b-449ac3c265ae admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:33.516397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2a9bb-043a-492e-8cae-bf3ff65345e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:33.517071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a9bb-043a-492e-8cae-bf3ff65345e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:33.517359 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a9bb-043a-492e-8cae-bf3ff65345e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:33.517637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a9bb-043a-492e-8cae-bf3ff65345e3 admin admin] PATH_INFO: `/v3/projects/d8b3a961e0cc4224988fbc599a9d66b6/users/00ac8f6353c841eaba1e152d927de90b/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:33.518697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58146a7-59f1-4209-93ab-314b35c2e2af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:33.519301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58146a7-59f1-4209-93ab-314b35c2e2af service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:33.519583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2a9bb-043a-492e-8cae-bf3ff65345e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b3a961e0cc4224988fbc599a9d66b6, user_id=00ac8f6353c841eaba1e152d927de90b, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:33.519689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58146a7-59f1-4209-93ab-314b35c2e2af service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:33.519858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58146a7-59f1-4209-93ab-314b35c2e2af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:33.522462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2a9bb-043a-492e-8cae-bf3ff65345e3 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:33.533861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1038/2111] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:33 2022] PUT /identity/v3/projects/d8b3a961e0cc4224988fbc599a9d66b6/users/00ac8f6353c841eaba1e152d927de90b/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:33.547722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b702ef-0cb4-4880-9caa-9524837d5eb0 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:33.548053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b702ef-0cb4-4880-9caa-9524837d5eb0 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:33.548365 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b702ef-0cb4-4880-9caa-9524837d5eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:33.578745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c1b702ef-0cb4-4880-9caa-9524837d5eb0 None None] MFA Rules not processed for user `00ac8f6353c841eaba1e152d927de90b`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:33.580727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c1b702ef-0cb4-4880-9caa-9524837d5eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:33.616042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58146a7-59f1-4209-93ab-314b35c2e2af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:33.617503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58146a7-59f1-4209-93ab-314b35c2e2af service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:33.629295 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1074/2112] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:33.662660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1039/2113] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:33.673835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58146a7-59f1-4209-93ab-314b35c2e2af service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:33.682915 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58146a7-59f1-4209-93ab-314b35c2e2af service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:33.694558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:33.738058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe6064-64b8-47f5-8a02-24598246e365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:33.738721 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe6064-64b8-47f5-8a02-24598246e365 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:33.739004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe6064-64b8-47f5-8a02-24598246e365 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:33.739282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe6064-64b8-47f5-8a02-24598246e365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:33.749952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe6064-64b8-47f5-8a02-24598246e365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:33.751484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe6064-64b8-47f5-8a02-24598246e365 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:33.762370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1075/2114] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:33 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:33.826356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b702ef-0cb4-4880-9caa-9524837d5eb0 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:33.837920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b702ef-0cb4-4880-9caa-9524837d5eb0 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:33.841560 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e94b2-0cd2-449c-ab93-b2427165353e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:33.842175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e94b2-0cd2-449c-ab93-b2427165353e service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:33.842461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e94b2-0cd2-449c-ab93-b2427165353e service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:33.842850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e94b2-0cd2-449c-ab93-b2427165353e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:33.902181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e94b2-0cd2-449c-ab93-b2427165353e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:33.903631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e94b2-0cd2-449c-ab93-b2427165353e service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:33.912638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1040/2115] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:33 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:34.214427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe6064-64b8-47f5-8a02-24598246e365 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:34.291827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe6064-64b8-47f5-8a02-24598246e365 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:34.296952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:34.297661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:34.297946 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:34.298245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] PATH_INFO: `/v3/users/5df2cf652d6f4f849de71502e8d03d02` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:34.299371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df2cf652d6f4f849de71502e8d03d02)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:34.301309 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:34.361902 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:34.574533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df2cf652d6f4f849de71502e8d03d02'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:34.613021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df2cf652d6f4f849de71502e8d03d02'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:34.614052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df2cf652d6f4f849de71502e8d03d02'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:34.614987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df2cf652d6f4f849de71502e8d03d02'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:34.688460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:34.759033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df2cf652d6f4f849de71502e8d03d02'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:34.772425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1076/2116] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:34 2022] DELETE /identity/v3/users/5df2cf652d6f4f849de71502e8d03d02 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:34.776879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34b20-b72a-40e7-a64b-a0bb418f428f None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:34.777506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34b20-b72a-40e7-a64b-a0bb418f428f None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:34.777788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34b20-b72a-40e7-a64b-a0bb418f428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:34.801058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-71b34b20-b72a-40e7-a64b-a0bb418f428f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:34.802322 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-71b34b20-b72a-40e7-a64b-a0bb418f428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:34.863691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1041/2117] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:34.872911 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:34.874330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:34.934044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50f9c0-ba40-4720-88a1-f58bcfd9d9a3 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:34.938072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a025d8b-356e-48e4-b004-697a3600732b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:34.938680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a025d8b-356e-48e4-b004-697a3600732b service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:34.938965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a025d8b-356e-48e4-b004-697a3600732b service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:34.939246 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a025d8b-356e-48e4-b004-697a3600732b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:34.948703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a025d8b-356e-48e4-b004-697a3600732b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:34.950144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a025d8b-356e-48e4-b004-697a3600732b service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:34.960988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1077/2118] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:35.304797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:35.405356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b34b20-b72a-40e7-a64b-a0bb418f428f None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:35.407324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-71b34b20-b72a-40e7-a64b-a0bb418f428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:35.423043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b34b20-b72a-40e7-a64b-a0bb418f428f None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:35.428510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:35.430010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:35.430394 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:35.430854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] PATH_INFO: `/v3/projects/60ee4c70bf8344b793bb990629c253f3` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:35.432132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60ee4c70bf8344b793bb990629c253f3)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:35.447130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:35.575407 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60ee4c70bf8344b793bb990629c253f3', 'request_id': 'req-825a7b1f-2c62-4d69-9486-8a2a27037124'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:35.581881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60ee4c70bf8344b793bb990629c253f3', 'request_id': 'req-825a7b1f-2c62-4d69-9486-8a2a27037124'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:35.589549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60ee4c70bf8344b793bb990629c253f3', 'request_id': 'req-825a7b1f-2c62-4d69-9486-8a2a27037124'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:35.756827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] The token cache is being invalidate because project 60ee4c70bf8344b793bb990629c253f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:35.757127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:35.766044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:35.766044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1042/2119] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:35 2022] DELETE /identity/v3/projects/60ee4c70bf8344b793bb990629c253f3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:35.768992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a025d8b-356e-48e4-b004-697a3600732b service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:35.779697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a025d8b-356e-48e4-b004-697a3600732b service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:35.852250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86b18c-969d-46fb-9262-d71b53dcad3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:35.853040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86b18c-969d-46fb-9262-d71b53dcad3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:35.853334 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86b18c-969d-46fb-9262-d71b53dcad3c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:35.853619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86b18c-969d-46fb-9262-d71b53dcad3c admin admin] PATH_INFO: `/v3/users/ec8ba80dbc1c4edd8d6aa3242db3b07f` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:35.854775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86b18c-969d-46fb-9262-d71b53dcad3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec8ba80dbc1c4edd8d6aa3242db3b07f)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:35.856796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86b18c-969d-46fb-9262-d71b53dcad3c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:35.959659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:36.065602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:36.196719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1a86b18c-969d-46fb-9262-d71b53dcad3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec8ba80dbc1c4edd8d6aa3242db3b07f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:36.211592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1a86b18c-969d-46fb-9262-d71b53dcad3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec8ba80dbc1c4edd8d6aa3242db3b07f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:36.212421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1a86b18c-969d-46fb-9262-d71b53dcad3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec8ba80dbc1c4edd8d6aa3242db3b07f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:36.213055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1a86b18c-969d-46fb-9262-d71b53dcad3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec8ba80dbc1c4edd8d6aa3242db3b07f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:36.295400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1a86b18c-969d-46fb-9262-d71b53dcad3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec8ba80dbc1c4edd8d6aa3242db3b07f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:36.312693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1078/2120] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:35 2022] DELETE /identity/v3/users/ec8ba80dbc1c4edd8d6aa3242db3b07f => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:36.731279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:36.883260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:36.884858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:36.898471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a7b1f-2c62-4d69-9486-8a2a27037124 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:36.961696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:36.962423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:36.962819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:36.963135 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] PATH_INFO: `/v3/projects/bd1e7104683a4b08a136a189cc97668c` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:36.964380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd1e7104683a4b08a136a189cc97668c)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:36.966394 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:37.013315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:37.093134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd1e7104683a4b08a136a189cc97668c', 'request_id': 'req-b2c57e16-5644-4a52-95f1-505f65f8cab0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:37.101955 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd1e7104683a4b08a136a189cc97668c', 'request_id': 'req-b2c57e16-5644-4a52-95f1-505f65f8cab0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:37.109074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd1e7104683a4b08a136a189cc97668c', 'request_id': 'req-b2c57e16-5644-4a52-95f1-505f65f8cab0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:37.192990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] The token cache is being invalidate because project bd1e7104683a4b08a136a189cc97668c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:37.194363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:37.194363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b2c57e16-5644-4a52-95f1-505f65f8cab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:37.195384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1043/2121] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:36 2022] DELETE /identity/v3/projects/bd1e7104683a4b08a136a189cc97668c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:37.201800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d42d8d-b80c-4d4b-9a99-90d4cf06f535 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:37.202275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d42d8d-b80c-4d4b-9a99-90d4cf06f535 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:37.202991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d42d8d-b80c-4d4b-9a99-90d4cf06f535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:37.224808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-b8d42d8d-b80c-4d4b-9a99-90d4cf06f535 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:37.226008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-b8d42d8d-b80c-4d4b-9a99-90d4cf06f535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:37.286838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1079/2122] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:37.295228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a44db-8e2e-4a8f-b786-0b343d8161b9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:37.297594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a44db-8e2e-4a8f-b786-0b343d8161b9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:37.297594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a44db-8e2e-4a8f-b786-0b343d8161b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:37.318058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-169a44db-8e2e-4a8f-b786-0b343d8161b9 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:37.319298 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-169a44db-8e2e-4a8f-b786-0b343d8161b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:37.328242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1044/2123] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:37.332632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d42d8d-b80c-4d4b-9a99-90d4cf06f535 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:37.344150 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d42d8d-b80c-4d4b-9a99-90d4cf06f535 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:37.349627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67452b-2dd7-4023-8794-ce90b7005095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:37.350293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67452b-2dd7-4023-8794-ce90b7005095 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:37.350718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67452b-2dd7-4023-8794-ce90b7005095 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:37.351070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67452b-2dd7-4023-8794-ce90b7005095 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:37.351650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67452b-2dd7-4023-8794-ce90b7005095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:37.352011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67452b-2dd7-4023-8794-ce90b7005095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:37.355086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67452b-2dd7-4023-8794-ce90b7005095 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:37.367448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1080/2124] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:51:37.372842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a44db-8e2e-4a8f-b786-0b343d8161b9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:37.385652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a44db-8e2e-4a8f-b786-0b343d8161b9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:37.391371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b437d93-1cc6-4a39-8bca-42aff33f879d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:37.392264 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437d93-1cc6-4a39-8bca-42aff33f879d admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:37.392559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437d93-1cc6-4a39-8bca-42aff33f879d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:37.392834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437d93-1cc6-4a39-8bca-42aff33f879d admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:37.393518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b437d93-1cc6-4a39-8bca-42aff33f879d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:37.397588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b437d93-1cc6-4a39-8bca-42aff33f879d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:37.456578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:37.503040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1045/2125] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:37 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:37.504250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67452b-2dd7-4023-8794-ce90b7005095 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:37.514809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67452b-2dd7-4023-8794-ce90b7005095 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:37.572502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25517470-6905-47c3-b27b-f48228c01fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:37.573116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25517470-6905-47c3-b27b-f48228c01fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:37.573419 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25517470-6905-47c3-b27b-f48228c01fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:37.573694 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25517470-6905-47c3-b27b-f48228c01fe1 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:37.574584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25517470-6905-47c3-b27b-f48228c01fe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:37.576354 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25517470-6905-47c3-b27b-f48228c01fe1 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:37.742516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1081/2126] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:37 2022] POST /identity/v3/users => generated 287 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:51:37.746607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b437d93-1cc6-4a39-8bca-42aff33f879d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:37.756846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b437d93-1cc6-4a39-8bca-42aff33f879d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:37.761385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b51e6-17c0-4565-a343-83b6d42e9499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:37.761982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b51e6-17c0-4565-a343-83b6d42e9499 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:37.762266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b51e6-17c0-4565-a343-83b6d42e9499 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:37.762541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b51e6-17c0-4565-a343-83b6d42e9499 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:37.763372 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b51e6-17c0-4565-a343-83b6d42e9499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:37.764991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b51e6-17c0-4565-a343-83b6d42e9499 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:37.774879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1046/2127] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:37.779068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25517470-6905-47c3-b27b-f48228c01fe1 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:37.788874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:37.790371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25517470-6905-47c3-b27b-f48228c01fe1 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:37.796141 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20de797-de4e-4e16-a34e-abfce8c6059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:37.798418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20de797-de4e-4e16-a34e-abfce8c6059f admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:37.798727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20de797-de4e-4e16-a34e-abfce8c6059f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:37.799010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20de797-de4e-4e16-a34e-abfce8c6059f admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:37.799672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20de797-de4e-4e16-a34e-abfce8c6059f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:37.802515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20de797-de4e-4e16-a34e-abfce8c6059f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:37.814265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1082/2128] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:37.818600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b51e6-17c0-4565-a343-83b6d42e9499 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:37.830959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b51e6-17c0-4565-a343-83b6d42e9499 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:37.836116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee990e6-3044-40cc-9413-60535d1ec198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:37.836786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee990e6-3044-40cc-9413-60535d1ec198 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:37.837126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee990e6-3044-40cc-9413-60535d1ec198 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:37.837728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee990e6-3044-40cc-9413-60535d1ec198 admin admin] PATH_INFO: `/v3/projects/7fd56f5ac6a84d2cbe5df806ec6f0848/users/bdba39bea160489db18cb424adc0d71c/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:37.855682 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee990e6-3044-40cc-9413-60535d1ec198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd56f5ac6a84d2cbe5df806ec6f0848, user_id=bdba39bea160489db18cb424adc0d71c, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:37.859010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee990e6-3044-40cc-9413-60535d1ec198 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:38.229629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1047/2129] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:37 2022] PUT /identity/v3/projects/7fd56f5ac6a84d2cbe5df806ec6f0848/users/bdba39bea160489db18cb424adc0d71c/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:38.234463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723604bf-b1f6-4089-bbdf-f63e5ddc6e47 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:38.234904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723604bf-b1f6-4089-bbdf-f63e5ddc6e47 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:38.235032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723604bf-b1f6-4089-bbdf-f63e5ddc6e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:38.259407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-723604bf-b1f6-4089-bbdf-f63e5ddc6e47 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:38.261808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-723604bf-b1f6-4089-bbdf-f63e5ddc6e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:38.343222 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1083/2130] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:38.353175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee990e6-3044-40cc-9413-60535d1ec198 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:38.356405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5ee990e6-3044-40cc-9413-60535d1ec198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:38.427036 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee990e6-3044-40cc-9413-60535d1ec198 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:38.427036 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13909f1-af30-4941-ac55-de02b42241fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:38.428121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13909f1-af30-4941-ac55-de02b42241fa service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:38.428523 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13909f1-af30-4941-ac55-de02b42241fa service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:38.428882 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13909f1-af30-4941-ac55-de02b42241fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:38.441549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13909f1-af30-4941-ac55-de02b42241fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:38.443344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13909f1-af30-4941-ac55-de02b42241fa service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:38.454970 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1048/2131] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:38.719225 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:38.818873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723604bf-b1f6-4089-bbdf-f63e5ddc6e47 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:38.820426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-723604bf-b1f6-4089-bbdf-f63e5ddc6e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:38.833347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723604bf-b1f6-4089-bbdf-f63e5ddc6e47 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:38.837997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:38.838620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:38.838935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:38.839240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] PATH_INFO: `/v3/users/cba52c999e414b96b5183cbc7ff9ea0f` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:38.840393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cba52c999e414b96b5183cbc7ff9ea0f)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:38.842346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:38.948952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:39.100048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cba52c999e414b96b5183cbc7ff9ea0f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.113282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cba52c999e414b96b5183cbc7ff9ea0f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.113966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cba52c999e414b96b5183cbc7ff9ea0f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.114594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cba52c999e414b96b5183cbc7ff9ea0f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.233094 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cba52c999e414b96b5183cbc7ff9ea0f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.246627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1084/2132] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:38 2022] DELETE /identity/v3/users/cba52c999e414b96b5183cbc7ff9ea0f => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:39.251366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07bef4-e73f-41e5-be47-d396ed0539af None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:39.251761 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07bef4-e73f-41e5-be47-d396ed0539af None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:39.251946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07bef4-e73f-41e5-be47-d396ed0539af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:39.271403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7d07bef4-e73f-41e5-be47-d396ed0539af None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:39.272656 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7d07bef4-e73f-41e5-be47-d396ed0539af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:39.333062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1049/2133] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:39.345457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:39.347264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:39.367175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07bef4-e73f-41e5-be47-d396ed0539af None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:39.368677 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7d07bef4-e73f-41e5-be47-d396ed0539af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:39.381691 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07bef4-e73f-41e5-be47-d396ed0539af None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:39.389645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ce1a8-bd96-4871-bce4-de486ed1ac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:39.390842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ce1a8-bd96-4871-bce4-de486ed1ac0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:39.391121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ce1a8-bd96-4871-bce4-de486ed1ac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:39.391502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ce1a8-bd96-4871-bce4-de486ed1ac0c admin admin] PATH_INFO: `/v3/users/4996594a6fe04173bbefb2b92e8ccb58` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:39.392767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ce1a8-bd96-4871-bce4-de486ed1ac0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4996594a6fe04173bbefb2b92e8ccb58)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:39.395019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ce1a8-bd96-4871-bce4-de486ed1ac0c admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:39.414985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaef1f5-80af-4c25-807c-989bf8213348 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:39.420463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927bbef-544b-4dac-9d70-1bffe529e9d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:39.421169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927bbef-544b-4dac-9d70-1bffe529e9d0 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:39.421462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927bbef-544b-4dac-9d70-1bffe529e9d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:39.421741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927bbef-544b-4dac-9d70-1bffe529e9d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:39.436695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927bbef-544b-4dac-9d70-1bffe529e9d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:39.438192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927bbef-544b-4dac-9d70-1bffe529e9d0 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:39.450536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1085/2134] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:51:39.458645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927bbef-544b-4dac-9d70-1bffe529e9d0 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:39.460243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c927bbef-544b-4dac-9d70-1bffe529e9d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:39.475838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927bbef-544b-4dac-9d70-1bffe529e9d0 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:39.482034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:39.482760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:39.483276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:39.483971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] PATH_INFO: `/v3/users/bcf2bbaf656147518cf40573f5f4e64c` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:39.485440 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf2bbaf656147518cf40573f5f4e64c)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:39.488257 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:39.623115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-010ce1a8-bd96-4871-bce4-de486ed1ac0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4996594a6fe04173bbefb2b92e8ccb58'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.624329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-010ce1a8-bd96-4871-bce4-de486ed1ac0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4996594a6fe04173bbefb2b92e8ccb58'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.636540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-010ce1a8-bd96-4871-bce4-de486ed1ac0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4996594a6fe04173bbefb2b92e8ccb58'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.787363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf2bbaf656147518cf40573f5f4e64c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.804682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf2bbaf656147518cf40573f5f4e64c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.806166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf2bbaf656147518cf40573f5f4e64c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.806891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf2bbaf656147518cf40573f5f4e64c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.828868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-010ce1a8-bd96-4871-bce4-de486ed1ac0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4996594a6fe04173bbefb2b92e8ccb58'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.835854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-010ce1a8-bd96-4871-bce4-de486ed1ac0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4996594a6fe04173bbefb2b92e8ccb58'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.842080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1050/2135] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:39 2022] DELETE /identity/v3/users/4996594a6fe04173bbefb2b92e8ccb58 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:39.846890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ee4a4-4ec1-463c-aea8-8ca3a69ad9c6 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:39.847348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ee4a4-4ec1-463c-aea8-8ca3a69ad9c6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:39.847719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ee4a4-4ec1-463c-aea8-8ca3a69ad9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:39.867723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-6b5ee4a4-4ec1-463c-aea8-8ca3a69ad9c6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:39.869055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6b5ee4a4-4ec1-463c-aea8-8ca3a69ad9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:39.925072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf2bbaf656147518cf40573f5f4e64c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:39.929474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1051/2136] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:39.944110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1086/2137] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:39 2022] DELETE /identity/v3/users/bcf2bbaf656147518cf40573f5f4e64c => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:39.950382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5780f4-8d0f-41ce-9cd5-c6f0c41d172c None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:39.950382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5780f4-8d0f-41ce-9cd5-c6f0c41d172c None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:39.951897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:39.952161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5780f4-8d0f-41ce-9cd5-c6f0c41d172c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:39.955342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:39.982074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-6e5780f4-8d0f-41ce-9cd5-c6f0c41d172c None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:39.983694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6e5780f4-8d0f-41ce-9cd5-c6f0c41d172c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:40.026688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538900c-7dbe-42a2-a92e-def5e2b57e4d admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:40.030676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54453c-6e77-470c-b97d-a0347f3fc5c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:40.031691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54453c-6e77-470c-b97d-a0347f3fc5c8 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:40.032273 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54453c-6e77-470c-b97d-a0347f3fc5c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:40.032722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54453c-6e77-470c-b97d-a0347f3fc5c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:40.056470 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1052/2138] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:40.057489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5780f4-8d0f-41ce-9cd5-c6f0c41d172c None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:40.061514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6e5780f4-8d0f-41ce-9cd5-c6f0c41d172c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:40.088318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5780f4-8d0f-41ce-9cd5-c6f0c41d172c None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:40.096638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:40.097491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:40.097941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:40.099185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] PATH_INFO: `/v3/projects/9bf0cb62f7794f328d483c2ea1f054e3` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:40.100632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf0cb62f7794f328d483c2ea1f054e3)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:40.103514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:40.137901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54453c-6e77-470c-b97d-a0347f3fc5c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:40.139375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54453c-6e77-470c-b97d-a0347f3fc5c8 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:40.151398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1087/2139] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:40.152231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54453c-6e77-470c-b97d-a0347f3fc5c8 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:40.154535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-fb54453c-6e77-470c-b97d-a0347f3fc5c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:40.168023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54453c-6e77-470c-b97d-a0347f3fc5c8 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:40.174590 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55caeee-0b1b-43b1-99e5-9b169d052828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:40.175376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55caeee-0b1b-43b1-99e5-9b169d052828 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:40.175662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55caeee-0b1b-43b1-99e5-9b169d052828 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:40.176054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55caeee-0b1b-43b1-99e5-9b169d052828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:40.190754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55caeee-0b1b-43b1-99e5-9b169d052828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:40.192261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55caeee-0b1b-43b1-99e5-9b169d052828 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:40.204361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1088/2140] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:40.207130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504f7dc-b694-4950-a27f-29a45081a7b9 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:40.207429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504f7dc-b694-4950-a27f-29a45081a7b9 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:40.208413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504f7dc-b694-4950-a27f-29a45081a7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:40.243450 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-a504f7dc-b694-4950-a27f-29a45081a7b9 None None] MFA Rules not processed for user `bdba39bea160489db18cb424adc0d71c`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:40.245887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a504f7dc-b694-4950-a27f-29a45081a7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:40.246623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf0cb62f7794f328d483c2ea1f054e3', 'request_id': 'req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:40.256152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf0cb62f7794f328d483c2ea1f054e3', 'request_id': 'req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:40.265478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf0cb62f7794f328d483c2ea1f054e3', 'request_id': 'req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:40.328725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1089/2141] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:40.340334 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504f7dc-b694-4950-a27f-29a45081a7b9 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:40.341878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a504f7dc-b694-4950-a27f-29a45081a7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:40.368735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] The token cache is being invalidate because project 9bf0cb62f7794f328d483c2ea1f054e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:40.369341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:40.370495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:40.372224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1053/2142] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:40 2022] DELETE /identity/v3/projects/9bf0cb62f7794f328d483c2ea1f054e3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:40.376906 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:40.379177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:40.412387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504f7dc-b694-4950-a27f-29a45081a7b9 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:40.418060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0e42f-11cf-4338-bc3c-5c43eee769b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:40.418779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0e42f-11cf-4338-bc3c-5c43eee769b2 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:40.419119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0e42f-11cf-4338-bc3c-5c43eee769b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:40.419464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0e42f-11cf-4338-bc3c-5c43eee769b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:40.420199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0e42f-11cf-4338-bc3c-5c43eee769b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:40.423204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0e42f-11cf-4338-bc3c-5c43eee769b2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:40.454532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bc632-4aaa-4393-b6cf-ca10c6f9978b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:40.459583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:40.462537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:40.462537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:40.462537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] PATH_INFO: `/v3/users/d5ee9970e6454e65a427932756786fb5` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:40.462537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5ee9970e6454e65a427932756786fb5)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:40.464302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:40.537911 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1090/2143] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:40 2022] POST /identity/v3/projects => generated 371 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:40.542113 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0e42f-11cf-4338-bc3c-5c43eee769b2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:40.544030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-13e0e42f-11cf-4338-bc3c-5c43eee769b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:40.628844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0e42f-11cf-4338-bc3c-5c43eee769b2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:40.635020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf3ffa-a0f0-4e8a-b843-a898bd12f2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:40.635657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf3ffa-a0f0-4e8a-b843-a898bd12f2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:40.643656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf3ffa-a0f0-4e8a-b843-a898bd12f2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:40.643656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf3ffa-a0f0-4e8a-b843-a898bd12f2c8 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:40.643656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf3ffa-a0f0-4e8a-b843-a898bd12f2c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:40.643656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf3ffa-a0f0-4e8a-b843-a898bd12f2c8 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:40.689161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5ee9970e6454e65a427932756786fb5'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:40.689917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5ee9970e6454e65a427932756786fb5'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:40.704029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5ee9970e6454e65a427932756786fb5'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:40.772406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1091/2144] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:40 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:51:40.774366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf3ffa-a0f0-4e8a-b843-a898bd12f2c8 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:40.792388 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf3ffa-a0f0-4e8a-b843-a898bd12f2c8 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:40.797463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5ee9970e6454e65a427932756786fb5'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:40.806222 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5ee9970e6454e65a427932756786fb5'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:40.816919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1054/2145] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:40 2022] DELETE /identity/v3/users/d5ee9970e6454e65a427932756786fb5 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:40.816919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:40.817367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:40.817543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279845a-a910-4056-9ea3-c93f4d9aa669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:40.818106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279845a-a910-4056-9ea3-c93f4d9aa669 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:40.818397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279845a-a910-4056-9ea3-c93f4d9aa669 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:40.818680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279845a-a910-4056-9ea3-c93f4d9aa669 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:40.819341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279845a-a910-4056-9ea3-c93f4d9aa669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:40.823079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279845a-a910-4056-9ea3-c93f4d9aa669 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:40.886786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:40.896870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1092/2146] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:51:40.897875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279845a-a910-4056-9ea3-c93f4d9aa669 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:40.909815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c279845a-a910-4056-9ea3-c93f4d9aa669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:40.943047 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279845a-a910-4056-9ea3-c93f4d9aa669 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:40.967512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f3f72-4f4b-4622-834e-24456473b280 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:40.973543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:40.974290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:40.974665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:40.975102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] PATH_INFO: `/v3/projects/8c136b19c69c47388515601c08251a27` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:40.976464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c136b19c69c47388515601c08251a27)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:40.978657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:41.030470 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:41.032658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:41.033153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:41.033272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] PATH_INFO: `/v3/projects/d6d3b23f9762469ea091237823aebc4b` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:41.034652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6d3b23f9762469ea091237823aebc4b)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:41.037100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:41.157377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c136b19c69c47388515601c08251a27', 'request_id': 'req-a459c2cb-d733-43e4-8828-97130fc89027'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:41.162901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c136b19c69c47388515601c08251a27', 'request_id': 'req-a459c2cb-d733-43e4-8828-97130fc89027'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:41.169354 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c136b19c69c47388515601c08251a27', 'request_id': 'req-a459c2cb-d733-43e4-8828-97130fc89027'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:41.179705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:41.205194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6d3b23f9762469ea091237823aebc4b', 'request_id': 'req-0edbd454-d8f4-4019-9775-71a19aed6b16'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:41.272600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] The token cache is being invalidate because project 8c136b19c69c47388515601c08251a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:41.273143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:41.273594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:41.275125 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1055/2147] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:40 2022] DELETE /identity/v3/projects/8c136b19c69c47388515601c08251a27 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:41.275889 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:41.277861 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:41.381872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459c2cb-d733-43e4-8828-97130fc89027 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:41.399049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6d3b23f9762469ea091237823aebc4b', 'request_id': 'req-0edbd454-d8f4-4019-9775-71a19aed6b16'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:41.406925 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6d3b23f9762469ea091237823aebc4b', 'request_id': 'req-0edbd454-d8f4-4019-9775-71a19aed6b16'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:41.419641 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] The token cache is being invalidate because project d6d3b23f9762469ea091237823aebc4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:41.420340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:41.421630 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0edbd454-d8f4-4019-9775-71a19aed6b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:41.423788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1093/2148] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:40 2022] DELETE /identity/v3/projects/d6d3b23f9762469ea091237823aebc4b => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:41.425322 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab44b7e-9981-4261-90fe-9228495c6da2 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:41.425871 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab44b7e-9981-4261-90fe-9228495c6da2 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:41.426391 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab44b7e-9981-4261-90fe-9228495c6da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:41.450217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1ab44b7e-9981-4261-90fe-9228495c6da2 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:41.452208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1ab44b7e-9981-4261-90fe-9228495c6da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:41.463855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67707270-67d9-4f84-a563-236cf1fd203e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:41.464584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67707270-67d9-4f84-a563-236cf1fd203e admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:41.464874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67707270-67d9-4f84-a563-236cf1fd203e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:41.465154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67707270-67d9-4f84-a563-236cf1fd203e admin admin] PATH_INFO: `/v3/projects/ab0e7db625ac43228178231236d03fd2/users/9aa89085efdc493099215b8222bd16f0/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:41.485730 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67707270-67d9-4f84-a563-236cf1fd203e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0e7db625ac43228178231236d03fd2, user_id=9aa89085efdc493099215b8222bd16f0, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:41.488609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67707270-67d9-4f84-a563-236cf1fd203e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:41.540361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1094/2149] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:41.549393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7317-14dd-4523-9588-e9e3c399c083 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:41.549722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7317-14dd-4523-9588-e9e3c399c083 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:41.549976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7317-14dd-4523-9588-e9e3c399c083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:41.571974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-8ebd7317-14dd-4523-9588-e9e3c399c083 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:41.573240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8ebd7317-14dd-4523-9588-e9e3c399c083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:41.581881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1095/2150] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:41.588990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd7317-14dd-4523-9588-e9e3c399c083 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:41.600381 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd7317-14dd-4523-9588-e9e3c399c083 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:41.605194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76aede0-8a28-42d9-bc6b-1e775f43a8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:41.605837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aede0-8a28-42d9-bc6b-1e775f43a8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:41.606127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aede0-8a28-42d9-bc6b-1e775f43a8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:41.606505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aede0-8a28-42d9-bc6b-1e775f43a8d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:41.607223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76aede0-8a28-42d9-bc6b-1e775f43a8d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:41.607542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76aede0-8a28-42d9-bc6b-1e775f43a8d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:41.609847 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76aede0-8a28-42d9-bc6b-1e775f43a8d7 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:41.631703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1096/2151] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:51:41.636739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76aede0-8a28-42d9-bc6b-1e775f43a8d7 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:41.638574 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-b76aede0-8a28-42d9-bc6b-1e775f43a8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:41.663375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76aede0-8a28-42d9-bc6b-1e775f43a8d7 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:41.670461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:41.672261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:41.685730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:41.685730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] PATH_INFO: `/v3/projects/ced93dd26ce64cf2a635078ba6341ccb` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:41.685730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ced93dd26ce64cf2a635078ba6341ccb)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:41.685730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:41.688669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1056/2152] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:41 2022] PUT /identity/v3/projects/ab0e7db625ac43228178231236d03fd2/users/9aa89085efdc493099215b8222bd16f0/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:41.688669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67707270-67d9-4f84-a563-236cf1fd203e admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:41.764525 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67707270-67d9-4f84-a563-236cf1fd203e admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:41.769951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cd46f-0184-4452-8bc7-930f84599493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:41.770500 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cd46f-0184-4452-8bc7-930f84599493 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:41.770811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cd46f-0184-4452-8bc7-930f84599493 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:41.771097 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cd46f-0184-4452-8bc7-930f84599493 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:41.771760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cd46f-0184-4452-8bc7-930f84599493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:41.773749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cd46f-0184-4452-8bc7-930f84599493 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:41.853125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ced93dd26ce64cf2a635078ba6341ccb', 'request_id': 'req-842d8814-53b2-4853-aa30-13c79e658a2f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.008471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1057/2153] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:41 2022] POST /identity/v3/projects => generated 383 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:42.008471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cd46f-0184-4452-8bc7-930f84599493 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:42.008471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f19cd46f-0184-4452-8bc7-930f84599493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:42.025110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ced93dd26ce64cf2a635078ba6341ccb', 'request_id': 'req-842d8814-53b2-4853-aa30-13c79e658a2f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.034040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ced93dd26ce64cf2a635078ba6341ccb', 'request_id': 'req-842d8814-53b2-4853-aa30-13c79e658a2f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.048159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] The token cache is being invalidate because project ced93dd26ce64cf2a635078ba6341ccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:42.048451 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.049179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.051129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1097/2154] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:41 2022] DELETE /identity/v3/projects/ced93dd26ce64cf2a635078ba6341ccb => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:42.052103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:42.057194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:42.084769 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cd46f-0184-4452-8bc7-930f84599493 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:42.092404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadbc69-b35a-40f7-be75-bb1d89f2d685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:42.095857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadbc69-b35a-40f7-be75-bb1d89f2d685 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:42.095857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadbc69-b35a-40f7-be75-bb1d89f2d685 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:42.095857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadbc69-b35a-40f7-be75-bb1d89f2d685 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:42.095857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadbc69-b35a-40f7-be75-bb1d89f2d685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:42.097616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadbc69-b35a-40f7-be75-bb1d89f2d685 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:42.110275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1058/2155] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:42.111528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadbc69-b35a-40f7-be75-bb1d89f2d685 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:42.113249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-fcadbc69-b35a-40f7-be75-bb1d89f2d685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:42.121831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d8814-53b2-4853-aa30-13c79e658a2f admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:42.127493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:42.130255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadbc69-b35a-40f7-be75-bb1d89f2d685 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:42.130363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:42.130363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:42.130363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] PATH_INFO: `/v3/users/97326c4e4181471fbaefb16427b8e50e` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:42.130363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97326c4e4181471fbaefb16427b8e50e)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:42.131793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:42.133016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c030ba0-90e2-4595-a8dc-16d46552a323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:42.133733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c030ba0-90e2-4595-a8dc-16d46552a323 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:42.134077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c030ba0-90e2-4595-a8dc-16d46552a323 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:42.134909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c030ba0-90e2-4595-a8dc-16d46552a323 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:42.135036 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c030ba0-90e2-4595-a8dc-16d46552a323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:42.136911 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c030ba0-90e2-4595-a8dc-16d46552a323 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:42.234857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1059/2156] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:42 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:42.235820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c030ba0-90e2-4595-a8dc-16d46552a323 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:42.238207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8c030ba0-90e2-4595-a8dc-16d46552a323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:42.251120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c030ba0-90e2-4595-a8dc-16d46552a323 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:42.256150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd5a83-7416-4dfb-9fa2-fdc99ed279a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:42.256789 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd5a83-7416-4dfb-9fa2-fdc99ed279a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:42.257082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd5a83-7416-4dfb-9fa2-fdc99ed279a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:42.257349 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd5a83-7416-4dfb-9fa2-fdc99ed279a6 admin admin] PATH_INFO: `/v3/users/2a55d8db5bb44aa78f2340ff18ad1c7c` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:42.258614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd5a83-7416-4dfb-9fa2-fdc99ed279a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a55d8db5bb44aa78f2340ff18ad1c7c)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:42.260593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd5a83-7416-4dfb-9fa2-fdc99ed279a6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:42.315805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97326c4e4181471fbaefb16427b8e50e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.329288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97326c4e4181471fbaefb16427b8e50e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.330010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97326c4e4181471fbaefb16427b8e50e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.330642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97326c4e4181471fbaefb16427b8e50e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.428090 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97326c4e4181471fbaefb16427b8e50e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.441435 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1098/2157] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:42 2022] DELETE /identity/v3/users/97326c4e4181471fbaefb16427b8e50e => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:42.442316 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:42.443667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:42.487658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6efd5a83-7416-4dfb-9fa2-fdc99ed279a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a55d8db5bb44aa78f2340ff18ad1c7c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.489154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6efd5a83-7416-4dfb-9fa2-fdc99ed279a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a55d8db5bb44aa78f2340ff18ad1c7c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.509724 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6efd5a83-7416-4dfb-9fa2-fdc99ed279a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a55d8db5bb44aa78f2340ff18ad1c7c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.514080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57782fd6-660a-4f39-8e37-07509a992a32 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:42.519649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4354811-9692-4f1f-91a5-1c15aca37c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:42.525270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4354811-9692-4f1f-91a5-1c15aca37c8a admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:42.525270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4354811-9692-4f1f-91a5-1c15aca37c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:42.525270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4354811-9692-4f1f-91a5-1c15aca37c8a admin admin] PATH_INFO: `/v3/projects/ab0e7db625ac43228178231236d03fd2/users/9aa89085efdc493099215b8222bd16f0/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:42.525270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4354811-9692-4f1f-91a5-1c15aca37c8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0e7db625ac43228178231236d03fd2, user_id=9aa89085efdc493099215b8222bd16f0, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:42.526031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4354811-9692-4f1f-91a5-1c15aca37c8a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:42.540864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1099/2158] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:42 2022] PUT /identity/v3/projects/ab0e7db625ac43228178231236d03fd2/users/9aa89085efdc493099215b8222bd16f0/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:42.541830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4354811-9692-4f1f-91a5-1c15aca37c8a admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:42.543498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c4354811-9692-4f1f-91a5-1c15aca37c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:42.594078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6efd5a83-7416-4dfb-9fa2-fdc99ed279a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a55d8db5bb44aa78f2340ff18ad1c7c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.605273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6efd5a83-7416-4dfb-9fa2-fdc99ed279a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a55d8db5bb44aa78f2340ff18ad1c7c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:42.611993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1060/2159] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:42 2022] DELETE /identity/v3/users/2a55d8db5bb44aa78f2340ff18ad1c7c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:42.613462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22561234-e28a-4783-ad66-208cf9667960 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:42.613818 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22561234-e28a-4783-ad66-208cf9667960 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:42.614166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22561234-e28a-4783-ad66-208cf9667960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:42.631447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4354811-9692-4f1f-91a5-1c15aca37c8a admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:42.651973 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-22561234-e28a-4783-ad66-208cf9667960 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:42.653444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-22561234-e28a-4783-ad66-208cf9667960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:42.718805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa562e59-ce66-4a71-8e76-8f90ee828866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:42.719324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa562e59-ce66-4a71-8e76-8f90ee828866 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:42.719614 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa562e59-ce66-4a71-8e76-8f90ee828866 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:42.719913 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa562e59-ce66-4a71-8e76-8f90ee828866 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:42.720846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa562e59-ce66-4a71-8e76-8f90ee828866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:42.723280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa562e59-ce66-4a71-8e76-8f90ee828866 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:42.738780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1100/2160] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:42.739645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa562e59-ce66-4a71-8e76-8f90ee828866 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:42.741550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-aa562e59-ce66-4a71-8e76-8f90ee828866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:42.753837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1061/2161] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:42.759804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa562e59-ce66-4a71-8e76-8f90ee828866 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:42.767452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee662a3-6f15-4cfb-b398-01e1d48f3720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:42.768455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee662a3-6f15-4cfb-b398-01e1d48f3720 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:42.768905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee662a3-6f15-4cfb-b398-01e1d48f3720 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:42.769262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee662a3-6f15-4cfb-b398-01e1d48f3720 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:42.770103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee662a3-6f15-4cfb-b398-01e1d48f3720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:42.772817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22561234-e28a-4783-ad66-208cf9667960 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:42.774314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-22561234-e28a-4783-ad66-208cf9667960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:42.775958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee662a3-6f15-4cfb-b398-01e1d48f3720 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:42.794488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1101/2162] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:42.799054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee662a3-6f15-4cfb-b398-01e1d48f3720 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:42.812997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee662a3-6f15-4cfb-b398-01e1d48f3720 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:42.818790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033c16-4422-4425-9d1e-4c0daecb5c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:42.819728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033c16-4422-4425-9d1e-4c0daecb5c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:42.820358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033c16-4422-4425-9d1e-4c0daecb5c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:42.820845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033c16-4422-4425-9d1e-4c0daecb5c1e admin admin] PATH_INFO: `/v3/projects/7a76c37d3f104207bfd653e68cdc1705/users/f1f9ffb090b340b58a92fca86fbd913e/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:42.840807 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00033c16-4422-4425-9d1e-4c0daecb5c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a76c37d3f104207bfd653e68cdc1705, user_id=f1f9ffb090b340b58a92fca86fbd913e, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:42.844057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00033c16-4422-4425-9d1e-4c0daecb5c1e admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:42.867825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22561234-e28a-4783-ad66-208cf9667960 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:42.872445 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197681d-392e-4902-abbf-9897b372a718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:42.873607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197681d-392e-4902-abbf-9897b372a718 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:42.873979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197681d-392e-4902-abbf-9897b372a718 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:42.876090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197681d-392e-4902-abbf-9897b372a718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:42.887372 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197681d-392e-4902-abbf-9897b372a718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:42.889573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197681d-392e-4902-abbf-9897b372a718 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:42.909306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1062/2163] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:51:42.943955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1102/2164] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:42 2022] PUT /identity/v3/projects/7a76c37d3f104207bfd653e68cdc1705/users/f1f9ffb090b340b58a92fca86fbd913e/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:42.945862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babc59b-2cb8-4bc2-b0eb-3a5a5f946dd0 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:42.946158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babc59b-2cb8-4bc2-b0eb-3a5a5f946dd0 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:42.946442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babc59b-2cb8-4bc2-b0eb-3a5a5f946dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:42.976849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-0babc59b-2cb8-4bc2-b0eb-3a5a5f946dd0 None None] MFA Rules not processed for user `f1f9ffb090b340b58a92fca86fbd913e`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:42.978158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0babc59b-2cb8-4bc2-b0eb-3a5a5f946dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:43.001367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:43.081661 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1063/2165] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:43.082966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033c16-4422-4425-9d1e-4c0daecb5c1e admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:43.183317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033c16-4422-4425-9d1e-4c0daecb5c1e admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:43.189478 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:43.193378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f92a5c-54f9-42ae-9db1-f70b6b58fc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:43.193933 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f92a5c-54f9-42ae-9db1-f70b6b58fc1f admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:43.194369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f92a5c-54f9-42ae-9db1-f70b6b58fc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:43.194679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f92a5c-54f9-42ae-9db1-f70b6b58fc1f admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:43.195722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f92a5c-54f9-42ae-9db1-f70b6b58fc1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:43.197977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f92a5c-54f9-42ae-9db1-f70b6b58fc1f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:43.310929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babc59b-2cb8-4bc2-b0eb-3a5a5f946dd0 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:43.319532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1103/2166] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:43 2022] POST /identity/v3/projects => generated 381 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:43.325251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babc59b-2cb8-4bc2-b0eb-3a5a5f946dd0 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:43.407957 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d5dae-237b-49de-88df-503d3295be88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:43.409013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d5dae-237b-49de-88df-503d3295be88 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:43.409303 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d5dae-237b-49de-88df-503d3295be88 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:43.409640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d5dae-237b-49de-88df-503d3295be88 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:43.410382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d5dae-237b-49de-88df-503d3295be88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:43.413349 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d5dae-237b-49de-88df-503d3295be88 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:43.476198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f92a5c-54f9-42ae-9db1-f70b6b58fc1f admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:43.483829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-65f92a5c-54f9-42ae-9db1-f70b6b58fc1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:43.503781 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f92a5c-54f9-42ae-9db1-f70b6b58fc1f admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:43.510636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:43.511428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:43.511824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:43.518135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] PATH_INFO: `/v3/projects/ea06c8e4c8414e90aa9dea9e2ca794b7` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:43.518135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea06c8e4c8414e90aa9dea9e2ca794b7)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:43.519751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:43.579085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1064/2167] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:43 2022] POST /identity/v3/users => generated 291 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:43.582558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d5dae-237b-49de-88df-503d3295be88 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:43.596647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d5dae-237b-49de-88df-503d3295be88 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:43.629662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd7187-0cce-4cbe-8dd6-b31f9eefe31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:43.630432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd7187-0cce-4cbe-8dd6-b31f9eefe31b admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:43.630792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd7187-0cce-4cbe-8dd6-b31f9eefe31b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:43.631154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd7187-0cce-4cbe-8dd6-b31f9eefe31b admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:43.632265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd7187-0cce-4cbe-8dd6-b31f9eefe31b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:43.634709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd7187-0cce-4cbe-8dd6-b31f9eefe31b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:43.651910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1065/2168] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:43.655087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd7187-0cce-4cbe-8dd6-b31f9eefe31b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:43.674821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd7187-0cce-4cbe-8dd6-b31f9eefe31b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:43.691696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184b07f-02da-46cd-b4d4-95d9ec48ec01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:43.692787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184b07f-02da-46cd-b4d4-95d9ec48ec01 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:43.693544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184b07f-02da-46cd-b4d4-95d9ec48ec01 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:43.697142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184b07f-02da-46cd-b4d4-95d9ec48ec01 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:43.697142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184b07f-02da-46cd-b4d4-95d9ec48ec01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:43.700985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184b07f-02da-46cd-b4d4-95d9ec48ec01 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:43.706935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea06c8e4c8414e90aa9dea9e2ca794b7', 'request_id': 'req-281f9b41-30af-431c-844d-257ee8e3b03a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:43.738193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1066/2169] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:51:43.738193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184b07f-02da-46cd-b4d4-95d9ec48ec01 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:43.749580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184b07f-02da-46cd-b4d4-95d9ec48ec01 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:43.839465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81eec7a-72c0-41d9-af01-667f7477edd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:43.840450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81eec7a-72c0-41d9-af01-667f7477edd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:43.841021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81eec7a-72c0-41d9-af01-667f7477edd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:43.841484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81eec7a-72c0-41d9-af01-667f7477edd1 admin admin] PATH_INFO: `/v3/projects/d731f1ff40a143ed803beb15506163c4/users/9bfa8830281e4416b7dc448092d145d8/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:43.861623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81eec7a-72c0-41d9-af01-667f7477edd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d731f1ff40a143ed803beb15506163c4, user_id=9bfa8830281e4416b7dc448092d145d8, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:43.864711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81eec7a-72c0-41d9-af01-667f7477edd1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:43.904687 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea06c8e4c8414e90aa9dea9e2ca794b7', 'request_id': 'req-281f9b41-30af-431c-844d-257ee8e3b03a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:43.912682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea06c8e4c8414e90aa9dea9e2ca794b7', 'request_id': 'req-281f9b41-30af-431c-844d-257ee8e3b03a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:43.924507 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] The token cache is being invalidate because project ea06c8e4c8414e90aa9dea9e2ca794b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:43.924874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:43.925629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-281f9b41-30af-431c-844d-257ee8e3b03a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:43.927314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1104/2170] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:43 2022] DELETE /identity/v3/projects/ea06c8e4c8414e90aa9dea9e2ca794b7 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:43.933237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201c90-2885-4fa7-8e3c-3c8a0940b1a0 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:43.933564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201c90-2885-4fa7-8e3c-3c8a0940b1a0 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:43.933912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201c90-2885-4fa7-8e3c-3c8a0940b1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:43.954947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-84201c90-2885-4fa7-8e3c-3c8a0940b1a0 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:43.956237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-84201c90-2885-4fa7-8e3c-3c8a0940b1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:43.964703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1105/2171] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:43.975215 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d62993-24fb-4cfa-a514-d187311823eb None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:43.975603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d62993-24fb-4cfa-a514-d187311823eb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:43.976091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d62993-24fb-4cfa-a514-d187311823eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:43.995878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-23d62993-24fb-4cfa-a514-d187311823eb None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:43.997127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-23d62993-24fb-4cfa-a514-d187311823eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:44.007165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1106/2172] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:44.011380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d62993-24fb-4cfa-a514-d187311823eb None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:44.020084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1067/2173] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:43 2022] PUT /identity/v3/projects/d731f1ff40a143ed803beb15506163c4/users/9bfa8830281e4416b7dc448092d145d8/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:44.022002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9c227-1816-4482-8be3-e55d53443a72 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:44.022293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9c227-1816-4482-8be3-e55d53443a72 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:44.022573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9c227-1816-4482-8be3-e55d53443a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:44.023175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d62993-24fb-4cfa-a514-d187311823eb None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:44.055167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-68f9c227-1816-4482-8be3-e55d53443a72 None None] MFA Rules not processed for user `9bfa8830281e4416b7dc448092d145d8`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:44.056443 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-68f9c227-1816-4482-8be3-e55d53443a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:44.105921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18df59-2858-4fe4-83e6-6b56a12fae81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:44.106965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18df59-2858-4fe4-83e6-6b56a12fae81 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:44.107478 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18df59-2858-4fe4-83e6-6b56a12fae81 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:44.108062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18df59-2858-4fe4-83e6-6b56a12fae81 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:44.109475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18df59-2858-4fe4-83e6-6b56a12fae81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:44.110008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18df59-2858-4fe4-83e6-6b56a12fae81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:44.112435 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18df59-2858-4fe4-83e6-6b56a12fae81 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:44.132799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1107/2174] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:51:44.133843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18df59-2858-4fe4-83e6-6b56a12fae81 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:44.145236 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1068/2175] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:44.146380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9c227-1816-4482-8be3-e55d53443a72 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:44.147383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18df59-2858-4fe4-83e6-6b56a12fae81 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:44.147835 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-68f9c227-1816-4482-8be3-e55d53443a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:44.152341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a507ca-468c-4c78-8315-61e52fd3dad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:44.152961 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a507ca-468c-4c78-8315-61e52fd3dad7 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:44.153248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a507ca-468c-4c78-8315-61e52fd3dad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:44.153524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a507ca-468c-4c78-8315-61e52fd3dad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:44.154152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a507ca-468c-4c78-8315-61e52fd3dad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:44.157011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a507ca-468c-4c78-8315-61e52fd3dad7 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:44.163103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9c227-1816-4482-8be3-e55d53443a72 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:44.168963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7378e-1de0-4fe6-b0c6-8325d4df48d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:44.169412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7378e-1de0-4fe6-b0c6-8325d4df48d3 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:44.173346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7378e-1de0-4fe6-b0c6-8325d4df48d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:44.173346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7378e-1de0-4fe6-b0c6-8325d4df48d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:44.173346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7378e-1de0-4fe6-b0c6-8325d4df48d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:44.173346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7378e-1de0-4fe6-b0c6-8325d4df48d3 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:44.366989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1069/2176] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:44 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:44.371257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7378e-1de0-4fe6-b0c6-8325d4df48d3 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:44.371553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1108/2177] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:44 2022] POST /identity/v3/projects => generated 377 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:51:44.372715 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a507ca-468c-4c78-8315-61e52fd3dad7 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:44.372968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d0c7378e-1de0-4fe6-b0c6-8325d4df48d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:44.441691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a507ca-468c-4c78-8315-61e52fd3dad7 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:44.446628 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde001fc-cd48-46b3-8630-f39e5a08d4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:44.447252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde001fc-cd48-46b3-8630-f39e5a08d4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:44.447542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde001fc-cd48-46b3-8630-f39e5a08d4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:44.447824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde001fc-cd48-46b3-8630-f39e5a08d4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:44.448509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde001fc-cd48-46b3-8630-f39e5a08d4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:44.450463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde001fc-cd48-46b3-8630-f39e5a08d4e8 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:44.454834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7378e-1de0-4fe6-b0c6-8325d4df48d3 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:44.460211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:44.461038 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:44.461515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:44.461916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] PATH_INFO: `/v3/projects/ffc0ffa8142e4165b7e9045e27fc7611` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:44.463098 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc0ffa8142e4165b7e9045e27fc7611)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:44.465130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:44.617843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1109/2178] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:44 2022] POST /identity/v3/users => generated 298 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:44.621944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde001fc-cd48-46b3-8630-f39e5a08d4e8 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:44.632588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde001fc-cd48-46b3-8630-f39e5a08d4e8 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:44.638060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b94c47-6a81-4111-af49-c5f26e03c1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:44.638214 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94c47-6a81-4111-af49-c5f26e03c1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:44.638531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94c47-6a81-4111-af49-c5f26e03c1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:44.638709 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94c47-6a81-4111-af49-c5f26e03c1b9 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:44.639491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b94c47-6a81-4111-af49-c5f26e03c1b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:44.641744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b94c47-6a81-4111-af49-c5f26e03c1b9 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:44.688944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc0ffa8142e4165b7e9045e27fc7611', 'request_id': 'req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:44.697661 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc0ffa8142e4165b7e9045e27fc7611', 'request_id': 'req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:44.709448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc0ffa8142e4165b7e9045e27fc7611', 'request_id': 'req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:44.759493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:44.791924 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1110/2179] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:44 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:44.793721 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b94c47-6a81-4111-af49-c5f26e03c1b9 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:44.806466 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b94c47-6a81-4111-af49-c5f26e03c1b9 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:44.821115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] The token cache is being invalidate because project ffc0ffa8142e4165b7e9045e27fc7611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:44.821115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:44.821959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:44.823826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1070/2180] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:44 2022] DELETE /identity/v3/projects/ffc0ffa8142e4165b7e9045e27fc7611 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:44.824452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:44.825880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:44.895848 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbd36c-d786-4824-8295-e4e7b328d804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:44.896513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbd36c-d786-4824-8295-e4e7b328d804 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:44.896802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbd36c-d786-4824-8295-e4e7b328d804 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:44.897081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbd36c-d786-4824-8295-e4e7b328d804 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:44.897750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbd36c-d786-4824-8295-e4e7b328d804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:44.899668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbd36c-d786-4824-8295-e4e7b328d804 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:44.904848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbe52e-2f89-4228-ace9-ba4a3ecdfe33 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:44.910662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d853ed-51d7-4152-97de-5349bf0228f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:44.911636 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d853ed-51d7-4152-97de-5349bf0228f3 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:44.911746 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d853ed-51d7-4152-97de-5349bf0228f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:44.912069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d853ed-51d7-4152-97de-5349bf0228f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:44.912751 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d853ed-51d7-4152-97de-5349bf0228f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:44.914393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d853ed-51d7-4152-97de-5349bf0228f3 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:44.918994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1111/2181] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:44.922401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbd36c-d786-4824-8295-e4e7b328d804 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:44.922401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-22dbd36c-d786-4824-8295-e4e7b328d804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:44.927707 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1071/2182] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:44.928474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d853ed-51d7-4152-97de-5349bf0228f3 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:44.931504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b8d853ed-51d7-4152-97de-5349bf0228f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:44.937838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbd36c-d786-4824-8295-e4e7b328d804 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:44.944831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:44.945523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:44.945813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:44.946251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] PATH_INFO: `/v3/users/00ac8f6353c841eaba1e152d927de90b` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:44.946737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d853ed-51d7-4152-97de-5349bf0228f3 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:44.947479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] RBAC: Authorizing `identity:delete_user(user_id=00ac8f6353c841eaba1e152d927de90b)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:44.949857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:44.952009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670ad51-ca23-4ff3-b6db-ed2d846b34c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:44.953020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670ad51-ca23-4ff3-b6db-ed2d846b34c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:44.953318 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670ad51-ca23-4ff3-b6db-ed2d846b34c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:44.953608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670ad51-ca23-4ff3-b6db-ed2d846b34c2 admin admin] PATH_INFO: `/v3/projects/3f35759c931d4afc885632207d8779bf/users/fa8bef1017f640b786b033a76c952de1/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:44.972955 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670ad51-ca23-4ff3-b6db-ed2d846b34c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f35759c931d4afc885632207d8779bf, user_id=fa8bef1017f640b786b033a76c952de1, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:44.975926 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670ad51-ca23-4ff3-b6db-ed2d846b34c2 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:45.024172 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:45.091611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1072/2183] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:44 2022] PUT /identity/v3/projects/3f35759c931d4afc885632207d8779bf/users/fa8bef1017f640b786b033a76c952de1/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:45.092742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670ad51-ca23-4ff3-b6db-ed2d846b34c2 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:45.189790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670ad51-ca23-4ff3-b6db-ed2d846b34c2 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:45.196418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bc87e-41b4-48a9-b209-6be14e3e09d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:45.197140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bc87e-41b4-48a9-b209-6be14e3e09d8 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:45.197845 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bc87e-41b4-48a9-b209-6be14e3e09d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:45.197956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bc87e-41b4-48a9-b209-6be14e3e09d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:45.199505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bc87e-41b4-48a9-b209-6be14e3e09d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:45.200507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bc87e-41b4-48a9-b209-6be14e3e09d8 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:45.212559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1073/2184] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:45.214209 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bc87e-41b4-48a9-b209-6be14e3e09d8 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:45.226054 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bc87e-41b4-48a9-b209-6be14e3e09d8 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:45.232812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a99e55-2bd3-432b-8d88-55561eee99bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:45.233069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a99e55-2bd3-432b-8d88-55561eee99bd admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:45.233279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a99e55-2bd3-432b-8d88-55561eee99bd admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:45.233643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a99e55-2bd3-432b-8d88-55561eee99bd admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:45.235151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a99e55-2bd3-432b-8d88-55561eee99bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:45.236687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a99e55-2bd3-432b-8d88-55561eee99bd admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:45.251707 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1074/2185] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:45.253178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3c03e-0489-4c8a-bc0a-74a7fe835890 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:45.255617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3c03e-0489-4c8a-bc0a-74a7fe835890 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:45.255617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3c03e-0489-4c8a-bc0a-74a7fe835890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:45.269931 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00ac8f6353c841eaba1e152d927de90b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:45.287915 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00ac8f6353c841eaba1e152d927de90b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:45.290068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00ac8f6353c841eaba1e152d927de90b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:45.290973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00ac8f6353c841eaba1e152d927de90b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:45.292927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-0ef3c03e-0489-4c8a-bc0a-74a7fe835890 None None] MFA Rules not processed for user `9aa89085efdc493099215b8222bd16f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:45.294517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0ef3c03e-0489-4c8a-bc0a-74a7fe835890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:45.348246 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00ac8f6353c841eaba1e152d927de90b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:45.363588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1112/2186] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:44 2022] DELETE /identity/v3/users/00ac8f6353c841eaba1e152d927de90b => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:45.364894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:45.366472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:45.390780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1075/2187] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:45.391582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3c03e-0489-4c8a-bc0a-74a7fe835890 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:45.393182 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0ef3c03e-0489-4c8a-bc0a-74a7fe835890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:45.441475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3c03e-0489-4c8a-bc0a-74a7fe835890 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:45.462808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184325f5-0c94-4fd4-97d9-4fc6312edbff admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:45.472730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3081e-81ed-43b5-a771-198ec1ce3c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:45.473839 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3081e-81ed-43b5-a771-198ec1ce3c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:45.474364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3081e-81ed-43b5-a771-198ec1ce3c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:45.474871 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3081e-81ed-43b5-a771-198ec1ce3c00 admin admin] PATH_INFO: `/v3/projects/0f8b180b7e7540358026bcdd76488da6/users/f725cad8009144f8b0fd3f3db0d71942/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:45.503632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3081e-81ed-43b5-a771-198ec1ce3c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8b180b7e7540358026bcdd76488da6, user_id=f725cad8009144f8b0fd3f3db0d71942, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:45.508200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3081e-81ed-43b5-a771-198ec1ce3c00 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:45.560602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0416623-1233-4028-8c13-65390b5375b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:45.561624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0416623-1233-4028-8c13-65390b5375b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:45.562228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0416623-1233-4028-8c13-65390b5375b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:45.562590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0416623-1233-4028-8c13-65390b5375b0 admin admin] PATH_INFO: `/v3/projects/3f35759c931d4afc885632207d8779bf/users/fa8bef1017f640b786b033a76c952de1/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:45.565217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0416623-1233-4028-8c13-65390b5375b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f35759c931d4afc885632207d8779bf, user_id=fa8bef1017f640b786b033a76c952de1, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:45.568511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0416623-1233-4028-8c13-65390b5375b0 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:45.580664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1076/2188] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:45 2022] PUT /identity/v3/projects/3f35759c931d4afc885632207d8779bf/users/fa8bef1017f640b786b033a76c952de1/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:45.581474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0416623-1233-4028-8c13-65390b5375b0 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:45.583277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d0416623-1233-4028-8c13-65390b5375b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:45.623169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1113/2189] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:45 2022] PUT /identity/v3/projects/0f8b180b7e7540358026bcdd76488da6/users/f725cad8009144f8b0fd3f3db0d71942/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:45.624581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fe90c-c16a-4168-b12a-06b848a8a68d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:45.624871 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fe90c-c16a-4168-b12a-06b848a8a68d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:45.629102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fe90c-c16a-4168-b12a-06b848a8a68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:45.731323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0416623-1233-4028-8c13-65390b5375b0 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:45.736321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3d1fe90c-c16a-4168-b12a-06b848a8a68d None None] MFA Rules not processed for user `fa8bef1017f640b786b033a76c952de1`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:45.737517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3d1fe90c-c16a-4168-b12a-06b848a8a68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:45.765662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:45.822076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995c2d6-575f-418e-a2e9-116562f7812f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:45.823087 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995c2d6-575f-418e-a2e9-116562f7812f service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:45.823544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995c2d6-575f-418e-a2e9-116562f7812f service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:45.824011 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995c2d6-575f-418e-a2e9-116562f7812f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:45.826853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f995c2d6-575f-418e-a2e9-116562f7812f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:45.877385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:45.885895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1114/2190] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:45.888111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f86fb4-d93b-4da3-bd12-ec2b6804b060 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:45.888579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f86fb4-d93b-4da3-bd12-ec2b6804b060 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:45.889351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f86fb4-d93b-4da3-bd12-ec2b6804b060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:45.912968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995c2d6-575f-418e-a2e9-116562f7812f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:45.915255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995c2d6-575f-418e-a2e9-116562f7812f service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:45.943625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1077/2191] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:45.944668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995c2d6-575f-418e-a2e9-116562f7812f service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:45.947169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-36f86fb4-d93b-4da3-bd12-ec2b6804b060 None None] MFA Rules not processed for user `f725cad8009144f8b0fd3f3db0d71942`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:45.948462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f995c2d6-575f-418e-a2e9-116562f7812f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:45.950622 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-36f86fb4-d93b-4da3-bd12-ec2b6804b060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:45.985943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995c2d6-575f-418e-a2e9-116562f7812f service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:46.016263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77041cd2-d856-4e02-8927-d286a332a5bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:46.016909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77041cd2-d856-4e02-8927-d286a332a5bf service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.017202 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77041cd2-d856-4e02-8927-d286a332a5bf service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.018978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77041cd2-d856-4e02-8927-d286a332a5bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.026584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-77041cd2-d856-4e02-8927-d286a332a5bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:46.098583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1115/2192] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:46.099796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f86fb4-d93b-4da3-bd12-ec2b6804b060 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:46.101595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-36f86fb4-d93b-4da3-bd12-ec2b6804b060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:46.177306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77041cd2-d856-4e02-8927-d286a332a5bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:46.179641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77041cd2-d856-4e02-8927-d286a332a5bf service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:46.198216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1078/2193] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:46.200147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77041cd2-d856-4e02-8927-d286a332a5bf service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:46.211425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f86fb4-d93b-4da3-bd12-ec2b6804b060 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:46.213521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77041cd2-d856-4e02-8927-d286a332a5bf service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:46.220248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:46.220943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.221241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.221525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] PATH_INFO: `/v3/projects/d8b3a961e0cc4224988fbc599a9d66b6` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.222714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b3a961e0cc4224988fbc599a9d66b6)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:46.226085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:46.233393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5755e9-d9fc-4ab1-9168-afc6be8c52f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:46.234543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5755e9-d9fc-4ab1-9168-afc6be8c52f8 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.234844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5755e9-d9fc-4ab1-9168-afc6be8c52f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.235129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5755e9-d9fc-4ab1-9168-afc6be8c52f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.235758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5755e9-d9fc-4ab1-9168-afc6be8c52f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:46.239036 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5755e9-d9fc-4ab1-9168-afc6be8c52f8 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:46.337254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1079/2194] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:46 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:46.338297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5755e9-d9fc-4ab1-9168-afc6be8c52f8 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:46.356077 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b3a961e0cc4224988fbc599a9d66b6', 'request_id': 'req-75141751-2062-4137-89cb-14f688b546b0'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:46.417971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5755e9-d9fc-4ab1-9168-afc6be8c52f8 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:46.422716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ad0f1-3046-4d10-a981-bf88dbd53351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:46.423330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ad0f1-3046-4d10-a981-bf88dbd53351 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.424218 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ad0f1-3046-4d10-a981-bf88dbd53351 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.424218 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ad0f1-3046-4d10-a981-bf88dbd53351 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.424589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ad0f1-3046-4d10-a981-bf88dbd53351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:46.426520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ad0f1-3046-4d10-a981-bf88dbd53351 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:46.476312 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b3a961e0cc4224988fbc599a9d66b6', 'request_id': 'req-75141751-2062-4137-89cb-14f688b546b0'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:46.483236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b3a961e0cc4224988fbc599a9d66b6', 'request_id': 'req-75141751-2062-4137-89cb-14f688b546b0'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:46.493563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] The token cache is being invalidate because project d8b3a961e0cc4224988fbc599a9d66b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:46.493854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:46.494520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:46.496066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1116/2195] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:46 2022] DELETE /identity/v3/projects/d8b3a961e0cc4224988fbc599a9d66b6 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:46.496832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:46.498253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:46.520188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1080/2196] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:46 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:46.521801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d754dc-f663-45dd-addd-b9da7d863b92 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.522161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d754dc-f663-45dd-addd-b9da7d863b92 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.522518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d754dc-f663-45dd-addd-b9da7d863b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.542114 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-52d754dc-f663-45dd-addd-b9da7d863b92 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:46.543332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-52d754dc-f663-45dd-addd-b9da7d863b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:46.551277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1081/2197] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:46.552168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d754dc-f663-45dd-addd-b9da7d863b92 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:46.553534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-52d754dc-f663-45dd-addd-b9da7d863b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:46.565936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d754dc-f663-45dd-addd-b9da7d863b92 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:46.568626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75141751-2062-4137-89cb-14f688b546b0 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:46.570586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3c8b5-50a1-480e-8622-834b05487448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:46.571189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3c8b5-50a1-480e-8622-834b05487448 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.571475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3c8b5-50a1-480e-8622-834b05487448 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.571778 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3c8b5-50a1-480e-8622-834b05487448 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.572476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3c8b5-50a1-480e-8622-834b05487448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:46.573673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f33dd9-8503-4855-8342-47974a8a5b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:46.574069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3c8b5-50a1-480e-8622-834b05487448 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:46.574306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f33dd9-8503-4855-8342-47974a8a5b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.574595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f33dd9-8503-4855-8342-47974a8a5b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.574876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f33dd9-8503-4855-8342-47974a8a5b02 admin admin] PATH_INFO: `/v3/users/72dc0c14d61e4bae9aaa31bf46cb1e08` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.576213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f33dd9-8503-4855-8342-47974a8a5b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72dc0c14d61e4bae9aaa31bf46cb1e08)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:46.578761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f33dd9-8503-4855-8342-47974a8a5b02 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:46.585316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1082/2198] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:46.590737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edbd94-1d0d-4bf4-a2b4-fef6d29dcdec None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.590737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edbd94-1d0d-4bf4-a2b4-fef6d29dcdec None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.590737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edbd94-1d0d-4bf4-a2b4-fef6d29dcdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.612415 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-88edbd94-1d0d-4bf4-a2b4-fef6d29dcdec None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:46.616700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-88edbd94-1d0d-4bf4-a2b4-fef6d29dcdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:46.621790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1083/2199] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:46.622596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edbd94-1d0d-4bf4-a2b4-fef6d29dcdec None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:46.632482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edbd94-1d0d-4bf4-a2b4-fef6d29dcdec None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:46.637161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689c308-d670-41fa-b738-334852074baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:46.637751 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689c308-d670-41fa-b738-334852074baa admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.638036 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689c308-d670-41fa-b738-334852074baa admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.638335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689c308-d670-41fa-b738-334852074baa admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.638953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689c308-d670-41fa-b738-334852074baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:46.641162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689c308-d670-41fa-b738-334852074baa admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:46.652784 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1084/2200] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:46.652784 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689c308-d670-41fa-b738-334852074baa admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:46.662435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689c308-d670-41fa-b738-334852074baa admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:46.667111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc8195-3b7b-4360-a0f2-460b046a0d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:46.667721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc8195-3b7b-4360-a0f2-460b046a0d0b admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.668101 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc8195-3b7b-4360-a0f2-460b046a0d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.668329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc8195-3b7b-4360-a0f2-460b046a0d0b admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.669054 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc8195-3b7b-4360-a0f2-460b046a0d0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:46.669360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc8195-3b7b-4360-a0f2-460b046a0d0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:46.670989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc8195-3b7b-4360-a0f2-460b046a0d0b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:46.684700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1085/2201] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:51:46.685543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc8195-3b7b-4360-a0f2-460b046a0d0b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:46.696762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc8195-3b7b-4360-a0f2-460b046a0d0b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:46.701739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509105e4-9d9b-4728-8fe8-8c5c03843b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:46.702390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509105e4-9d9b-4728-8fe8-8c5c03843b0e admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.703239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509105e4-9d9b-4728-8fe8-8c5c03843b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.703522 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509105e4-9d9b-4728-8fe8-8c5c03843b0e admin admin] PATH_INFO: `/v3/projects/2e687b6dcfa54cad860531f7599d5ced/users/2588e2fe19a3447e9b33850eba258a17/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.718379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509105e4-9d9b-4728-8fe8-8c5c03843b0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e687b6dcfa54cad860531f7599d5ced, user_id=2588e2fe19a3447e9b33850eba258a17, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:46.721082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509105e4-9d9b-4728-8fe8-8c5c03843b0e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:46.808862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1086/2202] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:46 2022] PUT /identity/v3/projects/2e687b6dcfa54cad860531f7599d5ced/users/2588e2fe19a3447e9b33850eba258a17/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:46.809770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509105e4-9d9b-4728-8fe8-8c5c03843b0e admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:46.817060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-53f33dd9-8503-4855-8342-47974a8a5b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72dc0c14d61e4bae9aaa31bf46cb1e08'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:46.830415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-53f33dd9-8503-4855-8342-47974a8a5b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72dc0c14d61e4bae9aaa31bf46cb1e08'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:46.831203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-53f33dd9-8503-4855-8342-47974a8a5b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72dc0c14d61e4bae9aaa31bf46cb1e08'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:46.831888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-53f33dd9-8503-4855-8342-47974a8a5b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72dc0c14d61e4bae9aaa31bf46cb1e08'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:46.878927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509105e4-9d9b-4728-8fe8-8c5c03843b0e admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:46.883782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6c577-f886-4636-baab-ab6aa4a5eab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:46.884423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6c577-f886-4636-baab-ab6aa4a5eab8 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.887780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6c577-f886-4636-baab-ab6aa4a5eab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.887780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6c577-f886-4636-baab-ab6aa4a5eab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.887780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6c577-f886-4636-baab-ab6aa4a5eab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:46.887780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6c577-f886-4636-baab-ab6aa4a5eab8 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:46.951068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-53f33dd9-8503-4855-8342-47974a8a5b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72dc0c14d61e4bae9aaa31bf46cb1e08'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:46.965780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1117/2203] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:46 2022] DELETE /identity/v3/users/72dc0c14d61e4bae9aaa31bf46cb1e08 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:46.967719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111d5a9-5bc2-43bb-b0cb-5760945e4d60 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:46.968990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111d5a9-5bc2-43bb-b0cb-5760945e4d60 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:46.969344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111d5a9-5bc2-43bb-b0cb-5760945e4d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:46.996049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1087/2204] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:46 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:46.996049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6c577-f886-4636-baab-ab6aa4a5eab8 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:46.996049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-48f6c577-f886-4636-baab-ab6aa4a5eab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:47.019601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f111d5a9-5bc2-43bb-b0cb-5760945e4d60 None None] MFA Rules not processed for user `2588e2fe19a3447e9b33850eba258a17`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:47.021480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f111d5a9-5bc2-43bb-b0cb-5760945e4d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:47.102216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6c577-f886-4636-baab-ab6aa4a5eab8 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:47.116704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9658-18b0-4617-baf7-b4930e81da96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:47.117349 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9658-18b0-4617-baf7-b4930e81da96 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.117639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9658-18b0-4617-baf7-b4930e81da96 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.117922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9658-18b0-4617-baf7-b4930e81da96 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.118574 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9658-18b0-4617-baf7-b4930e81da96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:47.123469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9658-18b0-4617-baf7-b4930e81da96 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:47.135562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1118/2205] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:47.139166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111d5a9-5bc2-43bb-b0cb-5760945e4d60 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:47.141722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f111d5a9-5bc2-43bb-b0cb-5760945e4d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:47.172266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111d5a9-5bc2-43bb-b0cb-5760945e4d60 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:47.172266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821632a1-50f1-4462-b451-aa8d79ec2a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:47.172266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821632a1-50f1-4462-b451-aa8d79ec2a12 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.172266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821632a1-50f1-4462-b451-aa8d79ec2a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.172620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821632a1-50f1-4462-b451-aa8d79ec2a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.172620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821632a1-50f1-4462-b451-aa8d79ec2a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:47.174278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821632a1-50f1-4462-b451-aa8d79ec2a12 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:47.215820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1088/2206] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:46 2022] POST /identity/v3/users => generated 292 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:51:47.220275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9658-18b0-4617-baf7-b4930e81da96 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:47.231292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9658-18b0-4617-baf7-b4930e81da96 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:47.236381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce6a81-b613-40b9-a2bf-e940e98a5ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:47.237108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce6a81-b613-40b9-a2bf-e940e98a5ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.237419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce6a81-b613-40b9-a2bf-e940e98a5ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.239974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce6a81-b613-40b9-a2bf-e940e98a5ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.239974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce6a81-b613-40b9-a2bf-e940e98a5ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:47.240161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce6a81-b613-40b9-a2bf-e940e98a5ab4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:47.256735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1089/2207] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:47.260783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce6a81-b613-40b9-a2bf-e940e98a5ab4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:47.272845 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce6a81-b613-40b9-a2bf-e940e98a5ab4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:47.279426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb7fb2-ca47-4598-a30a-6a5317701939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:47.280108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb7fb2-ca47-4598-a30a-6a5317701939 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.280398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb7fb2-ca47-4598-a30a-6a5317701939 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.280711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb7fb2-ca47-4598-a30a-6a5317701939 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.281381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb7fb2-ca47-4598-a30a-6a5317701939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:47.282998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb7fb2-ca47-4598-a30a-6a5317701939 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:47.296052 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1090/2208] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:51:47.298049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb7fb2-ca47-4598-a30a-6a5317701939 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:47.310185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb7fb2-ca47-4598-a30a-6a5317701939 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:47.315915 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1119/2209] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:47 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:47.317979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821632a1-50f1-4462-b451-aa8d79ec2a12 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:47.328171 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:47.390731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821632a1-50f1-4462-b451-aa8d79ec2a12 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:47.392465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203204d4-be0d-4bc1-9d67-4440b52c44a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:47.393138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203204d4-be0d-4bc1-9d67-4440b52c44a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.393424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203204d4-be0d-4bc1-9d67-4440b52c44a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.393700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203204d4-be0d-4bc1-9d67-4440b52c44a9 admin admin] PATH_INFO: `/v3/projects/f2520d8f6bbb42eab71dfe2a2b22180a/users/bb4a1efcfda24701b646f252b78ece4f/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.395960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27090d58-6538-4c0f-af66-9411e07f0d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:47.396607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090d58-6538-4c0f-af66-9411e07f0d46 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.397128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090d58-6538-4c0f-af66-9411e07f0d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.398151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090d58-6538-4c0f-af66-9411e07f0d46 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.398270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27090d58-6538-4c0f-af66-9411e07f0d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:47.400331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27090d58-6538-4c0f-af66-9411e07f0d46 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:47.409639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203204d4-be0d-4bc1-9d67-4440b52c44a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2520d8f6bbb42eab71dfe2a2b22180a, user_id=bb4a1efcfda24701b646f252b78ece4f, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:47.412553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203204d4-be0d-4bc1-9d67-4440b52c44a9 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:47.522951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1091/2210] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:47 2022] PUT /identity/v3/projects/f2520d8f6bbb42eab71dfe2a2b22180a/users/bb4a1efcfda24701b646f252b78ece4f/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:47.526149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1120/2211] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:47 2022] POST /identity/v3/users => generated 295 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:47.526257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203204d4-be0d-4bc1-9d67-4440b52c44a9 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:47.526257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-203204d4-be0d-4bc1-9d67-4440b52c44a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:47.527183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27090d58-6538-4c0f-af66-9411e07f0d46 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:47.537784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27090d58-6538-4c0f-af66-9411e07f0d46 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:47.600573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203204d4-be0d-4bc1-9d67-4440b52c44a9 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:47.600836 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f626f19-993a-4d30-b3d4-76abe3e92185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:47.601458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f626f19-993a-4d30-b3d4-76abe3e92185 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.601749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f626f19-993a-4d30-b3d4-76abe3e92185 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.602021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f626f19-993a-4d30-b3d4-76abe3e92185 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.602700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f626f19-993a-4d30-b3d4-76abe3e92185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:47.604428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f626f19-993a-4d30-b3d4-76abe3e92185 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:47.606137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:47.606902 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.607191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.607472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] PATH_INFO: `/v3/projects/35ef4da23ec44ed2a75e20f8cd365691` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.608692 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] RBAC: Authorizing `identity:delete_project(project_id=35ef4da23ec44ed2a75e20f8cd365691)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:47.611128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:47.615833 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1121/2212] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:47.617232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2201c97-18cc-4f11-9bd0-76c977033d30 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.618174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2201c97-18cc-4f11-9bd0-76c977033d30 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.618463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2201c97-18cc-4f11-9bd0-76c977033d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.648880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f2201c97-18cc-4f11-9bd0-76c977033d30 None None] MFA Rules not processed for user `bb4a1efcfda24701b646f252b78ece4f`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:47.650124 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f2201c97-18cc-4f11-9bd0-76c977033d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:47.717527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1122/2213] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:47.718420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2201c97-18cc-4f11-9bd0-76c977033d30 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:47.729534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2201c97-18cc-4f11-9bd0-76c977033d30 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:47.734850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5f2dc-11f4-425b-8a38-7f71578f1dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:47.735517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5f2dc-11f4-425b-8a38-7f71578f1dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.735802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5f2dc-11f4-425b-8a38-7f71578f1dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.736098 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5f2dc-11f4-425b-8a38-7f71578f1dd2 admin admin] PATH_INFO: `/v3/projects/e62572a469c8487088aade6d12c87daa/users/b8e4d5bd357f40bf9d54ec7b88602dea/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.752780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5f2dc-11f4-425b-8a38-7f71578f1dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62572a469c8487088aade6d12c87daa, user_id=b8e4d5bd357f40bf9d54ec7b88602dea, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:47.755524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5f2dc-11f4-425b-8a38-7f71578f1dd2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:47.758499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35ef4da23ec44ed2a75e20f8cd365691', 'request_id': 'req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:47.764196 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35ef4da23ec44ed2a75e20f8cd365691', 'request_id': 'req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:47.770396 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35ef4da23ec44ed2a75e20f8cd365691', 'request_id': 'req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:47.907749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] The token cache is being invalidate because project 35ef4da23ec44ed2a75e20f8cd365691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:47.908424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:47.908643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1123/2214] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:47 2022] PUT /identity/v3/projects/e62572a469c8487088aade6d12c87daa/users/b8e4d5bd357f40bf9d54ec7b88602dea/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:47.909032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:47.909474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5f2dc-11f4-425b-8a38-7f71578f1dd2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:47.910702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1092/2215] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:47 2022] DELETE /identity/v3/projects/35ef4da23ec44ed2a75e20f8cd365691 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:47.912643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:47.914017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:47.914366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4bb5f2dc-11f4-425b-8a38-7f71578f1dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:47.980463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5f2dc-11f4-425b-8a38-7f71578f1dd2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:47.980658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924e8ff-dce7-43f3-bfb9-c24b83b3771c admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:47.985702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffb278-2279-42b1-b79d-aaad883732c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:47.985851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26eae6-ba69-4dfe-88f4-8afad296e484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:47.986452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffb278-2279-42b1-b79d-aaad883732c7 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.986564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26eae6-ba69-4dfe-88f4-8afad296e484 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:47.986703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffb278-2279-42b1-b79d-aaad883732c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.986791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26eae6-ba69-4dfe-88f4-8afad296e484 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:47.987002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26eae6-ba69-4dfe-88f4-8afad296e484 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.987158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffb278-2279-42b1-b79d-aaad883732c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:47.987647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26eae6-ba69-4dfe-88f4-8afad296e484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:47.988128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffb278-2279-42b1-b79d-aaad883732c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:47.989275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26eae6-ba69-4dfe-88f4-8afad296e484 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:47.989716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:47.991969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffb278-2279-42b1-b79d-aaad883732c7 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:47.999731 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1093/2216] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:51:48.000852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9018b0-dc9a-4c36-8e9f-32edf683d3b4 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.001129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9018b0-dc9a-4c36-8e9f-32edf683d3b4 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.001407 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9018b0-dc9a-4c36-8e9f-32edf683d3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.021890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-6a9018b0-dc9a-4c36-8e9f-32edf683d3b4 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:48.023102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6a9018b0-dc9a-4c36-8e9f-32edf683d3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:48.033152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1094/2217] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:48.033968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9018b0-dc9a-4c36-8e9f-32edf683d3b4 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.044618 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9018b0-dc9a-4c36-8e9f-32edf683d3b4 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.051032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec194fb-8c68-499d-a8fc-db866c168a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.051725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec194fb-8c68-499d-a8fc-db866c168a64 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.052502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec194fb-8c68-499d-a8fc-db866c168a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.052801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec194fb-8c68-499d-a8fc-db866c168a64 admin admin] PATH_INFO: `/v3/projects/e62572a469c8487088aade6d12c87daa/users/b8e4d5bd357f40bf9d54ec7b88602dea/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.054967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec194fb-8c68-499d-a8fc-db866c168a64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62572a469c8487088aade6d12c87daa, user_id=b8e4d5bd357f40bf9d54ec7b88602dea, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.057978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec194fb-8c68-499d-a8fc-db866c168a64 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.069558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1095/2218] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:48 2022] PUT /identity/v3/projects/e62572a469c8487088aade6d12c87daa/users/b8e4d5bd357f40bf9d54ec7b88602dea/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:48.070803 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b606b-10b3-478b-b370-b44e19953209 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.071093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b606b-10b3-478b-b370-b44e19953209 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.071377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b606b-10b3-478b-b370-b44e19953209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.091166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-941b606b-10b3-478b-b370-b44e19953209 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:48.092441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-941b606b-10b3-478b-b370-b44e19953209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:48.092576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1124/2219] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:47 2022] POST /identity/v3/projects => generated 381 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:48.094156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffb278-2279-42b1-b79d-aaad883732c7 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.095935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-11ffb278-2279-42b1-b79d-aaad883732c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:48.156739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1096/2220] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:48.158094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dd2fc-23b7-4c6a-9861-b38f809da0b4 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.158387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dd2fc-23b7-4c6a-9861-b38f809da0b4 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.158490 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffb278-2279-42b1-b79d-aaad883732c7 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.158669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dd2fc-23b7-4c6a-9861-b38f809da0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.162693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7dcaa-3877-485a-b6e7-342e2f864792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.163314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7dcaa-3877-485a-b6e7-342e2f864792 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.163594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7dcaa-3877-485a-b6e7-342e2f864792 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.163882 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7dcaa-3877-485a-b6e7-342e2f864792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.166545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cfb7dcaa-3877-485a-b6e7-342e2f864792 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:48.190626 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-561dd2fc-23b7-4c6a-9861-b38f809da0b4 None None] MFA Rules not processed for user `b8e4d5bd357f40bf9d54ec7b88602dea`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:48.191936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-561dd2fc-23b7-4c6a-9861-b38f809da0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:48.273188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1097/2221] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:48.274140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561dd2fc-23b7-4c6a-9861-b38f809da0b4 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.287243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561dd2fc-23b7-4c6a-9861-b38f809da0b4 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.293102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca916f6-090f-44b6-a6d1-ce7cc5e5ae55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.293871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca916f6-090f-44b6-a6d1-ce7cc5e5ae55 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.294203 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca916f6-090f-44b6-a6d1-ce7cc5e5ae55 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.294485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca916f6-090f-44b6-a6d1-ce7cc5e5ae55 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.295233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca916f6-090f-44b6-a6d1-ce7cc5e5ae55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.297518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca916f6-090f-44b6-a6d1-ce7cc5e5ae55 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.365135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7dcaa-3877-485a-b6e7-342e2f864792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.366594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7dcaa-3877-485a-b6e7-342e2f864792 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.377843 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1098/2222] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:48 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:48.378055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1125/2223] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:48.378780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7dcaa-3877-485a-b6e7-342e2f864792 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.379485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca916f6-090f-44b6-a6d1-ce7cc5e5ae55 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.381532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9ca916f6-090f-44b6-a6d1-ce7cc5e5ae55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:48.389525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7dcaa-3877-485a-b6e7-342e2f864792 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.394431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49943c4e-42e0-48e0-8a02-5ad7771e0767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.395064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49943c4e-42e0-48e0-8a02-5ad7771e0767 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.395496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49943c4e-42e0-48e0-8a02-5ad7771e0767 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.395807 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49943c4e-42e0-48e0-8a02-5ad7771e0767 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.396508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49943c4e-42e0-48e0-8a02-5ad7771e0767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:48.396819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49943c4e-42e0-48e0-8a02-5ad7771e0767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.398434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49943c4e-42e0-48e0-8a02-5ad7771e0767 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.399529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca916f6-090f-44b6-a6d1-ce7cc5e5ae55 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.403946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c1b22-8fac-4ce7-85e0-d3a562d04892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.404579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c1b22-8fac-4ce7-85e0-d3a562d04892 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.404867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c1b22-8fac-4ce7-85e0-d3a562d04892 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.405146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c1b22-8fac-4ce7-85e0-d3a562d04892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.407206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c45c1b22-8fac-4ce7-85e0-d3a562d04892 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:48.416281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1126/2224] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:51:48.416281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49943c4e-42e0-48e0-8a02-5ad7771e0767 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.427390 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49943c4e-42e0-48e0-8a02-5ad7771e0767 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.435811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa61c90-9f8f-43e1-a2f2-631b2e3dbd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.436712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa61c90-9f8f-43e1-a2f2-631b2e3dbd06 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.437580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa61c90-9f8f-43e1-a2f2-631b2e3dbd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.437861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa61c90-9f8f-43e1-a2f2-631b2e3dbd06 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.438581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa61c90-9f8f-43e1-a2f2-631b2e3dbd06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.440272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa61c90-9f8f-43e1-a2f2-631b2e3dbd06 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.455936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1127/2225] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:48.455936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa61c90-9f8f-43e1-a2f2-631b2e3dbd06 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.466063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa61c90-9f8f-43e1-a2f2-631b2e3dbd06 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.471609 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ead9-d183-4b50-acba-a2762a21cd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.473139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ead9-d183-4b50-acba-a2762a21cd9c admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.473139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ead9-d183-4b50-acba-a2762a21cd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.473139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ead9-d183-4b50-acba-a2762a21cd9c admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.473566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52ead9-d183-4b50-acba-a2762a21cd9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.476415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52ead9-d183-4b50-acba-a2762a21cd9c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.487781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c1b22-8fac-4ce7-85e0-d3a562d04892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.489503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c1b22-8fac-4ce7-85e0-d3a562d04892 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.502601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1099/2226] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:48.503798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c1b22-8fac-4ce7-85e0-d3a562d04892 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.518806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c1b22-8fac-4ce7-85e0-d3a562d04892 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.518806 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc3662-814d-443a-afaf-669a4c162f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.524504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc3662-814d-443a-afaf-669a4c162f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.524762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc3662-814d-443a-afaf-669a4c162f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.525039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc3662-814d-443a-afaf-669a4c162f85 admin admin] PATH_INFO: `/v3/projects/7fea64ed980e40f8990093f9b717c6dc/users/a0e1407cb1864ac691cb87d80448a571/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.545332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc3662-814d-443a-afaf-669a4c162f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fea64ed980e40f8990093f9b717c6dc, user_id=a0e1407cb1864ac691cb87d80448a571, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.548432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc3662-814d-443a-afaf-669a4c162f85 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.596553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1128/2227] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:48 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:48.597567 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ead9-d183-4b50-acba-a2762a21cd9c admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.614950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1100/2228] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:48 2022] PUT /identity/v3/projects/7fea64ed980e40f8990093f9b717c6dc/users/a0e1407cb1864ac691cb87d80448a571/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:48.620384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc3662-814d-443a-afaf-669a4c162f85 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.677297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ead9-d183-4b50-acba-a2762a21cd9c admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.689255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc3662-814d-443a-afaf-669a4c162f85 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.694003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622f042-e2b5-4db6-9f97-787ab9bd712c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.694628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622f042-e2b5-4db6-9f97-787ab9bd712c admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.694977 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622f042-e2b5-4db6-9f97-787ab9bd712c admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.695227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622f042-e2b5-4db6-9f97-787ab9bd712c admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.695919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622f042-e2b5-4db6-9f97-787ab9bd712c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.697570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622f042-e2b5-4db6-9f97-787ab9bd712c admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.711364 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1101/2229] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:48.712461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622f042-e2b5-4db6-9f97-787ab9bd712c admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.723140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622f042-e2b5-4db6-9f97-787ab9bd712c admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.728200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae3327-9b3b-4089-b110-0508d6b47359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.728933 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3327-9b3b-4089-b110-0508d6b47359 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.729844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3327-9b3b-4089-b110-0508d6b47359 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.729844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3327-9b3b-4089-b110-0508d6b47359 admin admin] PATH_INFO: `/v3/projects/7fea64ed980e40f8990093f9b717c6dc/users/a0e1407cb1864ac691cb87d80448a571/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.731742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae3327-9b3b-4089-b110-0508d6b47359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fea64ed980e40f8990093f9b717c6dc, user_id=a0e1407cb1864ac691cb87d80448a571, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.735639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae3327-9b3b-4089-b110-0508d6b47359 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.745237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a605ed-6818-48a2-9db8-f555e3e233d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.746178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a605ed-6818-48a2-9db8-f555e3e233d2 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.746354 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a605ed-6818-48a2-9db8-f555e3e233d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.746659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a605ed-6818-48a2-9db8-f555e3e233d2 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.747307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a605ed-6818-48a2-9db8-f555e3e233d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.747929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1102/2230] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:48 2022] PUT /identity/v3/projects/7fea64ed980e40f8990093f9b717c6dc/users/a0e1407cb1864ac691cb87d80448a571/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:48.749292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a605ed-6818-48a2-9db8-f555e3e233d2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.763155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbbd1e-9244-499a-969d-04aaf4e4ef12 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.763155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbbd1e-9244-499a-969d-04aaf4e4ef12 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.763155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbbd1e-9244-499a-969d-04aaf4e4ef12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.763155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:48.790386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c9fbbd1e-9244-499a-969d-04aaf4e4ef12 None None] MFA Rules not processed for user `a0e1407cb1864ac691cb87d80448a571`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:48.791642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c9fbbd1e-9244-499a-969d-04aaf4e4ef12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:48.862574 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1103/2231] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:48.887405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1129/2232] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:48 2022] POST /identity/v3/users => generated 293 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:48.891411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbbd1e-9244-499a-969d-04aaf4e4ef12 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.903332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a605ed-6818-48a2-9db8-f555e3e233d2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.958171 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbbd1e-9244-499a-969d-04aaf4e4ef12 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.959720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a605ed-6818-48a2-9db8-f555e3e233d2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.963287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7a602-fc6a-4f16-aba0-943f5a8d381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:48.963940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7a602-fc6a-4f16-aba0-943f5a8d381d admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:48.964231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7a602-fc6a-4f16-aba0-943f5a8d381d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:48.964508 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7a602-fc6a-4f16-aba0-943f5a8d381d admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:48.965166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7a602-fc6a-4f16-aba0-943f5a8d381d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:48.966770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7a602-fc6a-4f16-aba0-943f5a8d381d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:48.977838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1104/2233] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:48.981792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7a602-fc6a-4f16-aba0-943f5a8d381d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:48.994330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7a602-fc6a-4f16-aba0-943f5a8d381d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:48.999546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeadb0b-a0bb-41bd-a563-ac3690e556f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:49.000907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeadb0b-a0bb-41bd-a563-ac3690e556f1 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:49.000907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeadb0b-a0bb-41bd-a563-ac3690e556f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:49.001095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeadb0b-a0bb-41bd-a563-ac3690e556f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:49.001911 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeadb0b-a0bb-41bd-a563-ac3690e556f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:49.003565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeadb0b-a0bb-41bd-a563-ac3690e556f1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:49.013711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee921634-97ac-4a35-a5d9-3a939bd4f426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:49.014253 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921634-97ac-4a35-a5d9-3a939bd4f426 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:49.014537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921634-97ac-4a35-a5d9-3a939bd4f426 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:49.014813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921634-97ac-4a35-a5d9-3a939bd4f426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:49.015511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1105/2234] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:51:49.019767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeadb0b-a0bb-41bd-a563-ac3690e556f1 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:49.025313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee921634-97ac-4a35-a5d9-3a939bd4f426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:49.026783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee921634-97ac-4a35-a5d9-3a939bd4f426 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:49.030179 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeadb0b-a0bb-41bd-a563-ac3690e556f1 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:49.035816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eacda7-7c58-41cb-a768-179cd6bd1f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:49.036516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eacda7-7c58-41cb-a768-179cd6bd1f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:49.036801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eacda7-7c58-41cb-a768-179cd6bd1f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:49.037136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eacda7-7c58-41cb-a768-179cd6bd1f17 admin admin] PATH_INFO: `/v3/projects/f40d6476c750438c9c09107c7757fa95/users/9348310fd06b473786d1c6be4f20c481/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:49.038127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1130/2235] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:51:49.054332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eacda7-7c58-41cb-a768-179cd6bd1f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40d6476c750438c9c09107c7757fa95, user_id=9348310fd06b473786d1c6be4f20c481, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:49.057199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eacda7-7c58-41cb-a768-179cd6bd1f17 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:49.129427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1106/2236] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:49 2022] PUT /identity/v3/projects/f40d6476c750438c9c09107c7757fa95/users/9348310fd06b473786d1c6be4f20c481/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:51:49.137114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ff9e6-9c3b-47d6-b68a-936b58d6b8bb None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:49.137616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ff9e6-9c3b-47d6-b68a-936b58d6b8bb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:49.137899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ff9e6-9c3b-47d6-b68a-936b58d6b8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:49.145141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eacda7-7c58-41cb-a768-179cd6bd1f17 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:49.155012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:49.161923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9a1ff9e6-9c3b-47d6-b68a-936b58d6b8bb None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:49.167884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9a1ff9e6-9c3b-47d6-b68a-936b58d6b8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:49.212107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eacda7-7c58-41cb-a768-179cd6bd1f17 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:49.216633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70355eab-1b7e-4556-ba67-ec6003e9952b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:49.217173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70355eab-1b7e-4556-ba67-ec6003e9952b service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:49.217452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70355eab-1b7e-4556-ba67-ec6003e9952b service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:49.217724 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70355eab-1b7e-4556-ba67-ec6003e9952b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:49.220323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-70355eab-1b7e-4556-ba67-ec6003e9952b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:49.237575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1131/2237] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:49.249050 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ff9e6-9c3b-47d6-b68a-936b58d6b8bb None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:49.252476 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9a1ff9e6-9c3b-47d6-b68a-936b58d6b8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:49.312343 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70355eab-1b7e-4556-ba67-ec6003e9952b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:49.315197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70355eab-1b7e-4556-ba67-ec6003e9952b service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:49.325785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1107/2238] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:49.336527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ff9e6-9c3b-47d6-b68a-936b58d6b8bb None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:49.340537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ec30-cd3f-4d90-b5bc-7cb6c3390638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:49.341069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ec30-cd3f-4d90-b5bc-7cb6c3390638 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:49.341356 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ec30-cd3f-4d90-b5bc-7cb6c3390638 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:49.341633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ec30-cd3f-4d90-b5bc-7cb6c3390638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:49.352761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74ec30-cd3f-4d90-b5bc-7cb6c3390638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:49.354464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74ec30-cd3f-4d90-b5bc-7cb6c3390638 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:49.365611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1132/2239] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:49.614639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70355eab-1b7e-4556-ba67-ec6003e9952b service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:49.624003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70355eab-1b7e-4556-ba67-ec6003e9952b service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:49.628493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513acf1-ebf3-428b-9e18-9ab9b5f8dd6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:49.629271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513acf1-ebf3-428b-9e18-9ab9b5f8dd6c service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:49.629553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513acf1-ebf3-428b-9e18-9ab9b5f8dd6c service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:49.629831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513acf1-ebf3-428b-9e18-9ab9b5f8dd6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:49.631751 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c513acf1-ebf3-428b-9e18-9ab9b5f8dd6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:49.705496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513acf1-ebf3-428b-9e18-9ab9b5f8dd6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:49.707673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513acf1-ebf3-428b-9e18-9ab9b5f8dd6c service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:49.717733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1108/2240] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:50.716932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:51.081080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ec30-cd3f-4d90-b5bc-7cb6c3390638 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:51.090088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ec30-cd3f-4d90-b5bc-7cb6c3390638 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:51.094471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd96fd-490d-41cc-8d76-8a4f15e04349 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:51.095067 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd96fd-490d-41cc-8d76-8a4f15e04349 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:51.095353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd96fd-490d-41cc-8d76-8a4f15e04349 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:51.095629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd96fd-490d-41cc-8d76-8a4f15e04349 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:51.097648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f4fd96fd-490d-41cc-8d76-8a4f15e04349 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:51.159687 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd96fd-490d-41cc-8d76-8a4f15e04349 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:51.161274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd96fd-490d-41cc-8d76-8a4f15e04349 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:51.172173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1133/2241] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:51.614588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:52.581310 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513acf1-ebf3-428b-9e18-9ab9b5f8dd6c service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:52.592511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513acf1-ebf3-428b-9e18-9ab9b5f8dd6c service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:52.598532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b182fba-8bf0-4791-98c5-633d09a59b57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:52.601200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b182fba-8bf0-4791-98c5-633d09a59b57 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:52.601792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b182fba-8bf0-4791-98c5-633d09a59b57 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:52.602276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b182fba-8bf0-4791-98c5-633d09a59b57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:52.607923 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5b182fba-8bf0-4791-98c5-633d09a59b57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:52.708591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b182fba-8bf0-4791-98c5-633d09a59b57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:52.710089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b182fba-8bf0-4791-98c5-633d09a59b57 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:52.721336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1109/2242] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:53.268019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:53.359584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:53.580881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd96fd-490d-41cc-8d76-8a4f15e04349 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:53.591858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd96fd-490d-41cc-8d76-8a4f15e04349 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:53.606497 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd97315-ffbb-4cd5-8351-f85e71285eac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:53.607458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd97315-ffbb-4cd5-8351-f85e71285eac service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:53.607910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd97315-ffbb-4cd5-8351-f85e71285eac service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:53.608285 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd97315-ffbb-4cd5-8351-f85e71285eac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:53.711801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd97315-ffbb-4cd5-8351-f85e71285eac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:53.713421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd97315-ffbb-4cd5-8351-f85e71285eac service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:53.724526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1134/2243] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:55.132158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:55.184732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b182fba-8bf0-4791-98c5-633d09a59b57 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:55.198501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b182fba-8bf0-4791-98c5-633d09a59b57 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:55.210340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:55.211134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:55.211517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:55.212081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] PATH_INFO: `/v3/users/bb4a1efcfda24701b646f252b78ece4f` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:55.213810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb4a1efcfda24701b646f252b78ece4f)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:55.217903 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:55.221154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:55.575405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb4a1efcfda24701b646f252b78ece4f'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:55.584092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb4a1efcfda24701b646f252b78ece4f'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:55.654021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb4a1efcfda24701b646f252b78ece4f'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:55.771424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb4a1efcfda24701b646f252b78ece4f'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:55.783268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb4a1efcfda24701b646f252b78ece4f'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:55.788232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1110/2244] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:55 2022] DELETE /identity/v3/users/bb4a1efcfda24701b646f252b78ece4f => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:55.792689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a1fc-1ee8-443c-9b05-da6bdc17bf9d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:55.793008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a1fc-1ee8-443c-9b05-da6bdc17bf9d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:55.793293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a1fc-1ee8-443c-9b05-da6bdc17bf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:55.817490 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0333a1fc-1ee8-443c-9b05-da6bdc17bf9d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:55.818790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0333a1fc-1ee8-443c-9b05-da6bdc17bf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:55.938553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1135/2245] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:55.981747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:55.983770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:56.092614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cb4f4-c0eb-4853-b88b-64ada49499e9 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:56.098063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c82657-9340-41a3-bb39-5a7826ea51fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:56.098916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c82657-9340-41a3-bb39-5a7826ea51fc service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:56.099213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c82657-9340-41a3-bb39-5a7826ea51fc service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:56.099554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c82657-9340-41a3-bb39-5a7826ea51fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:56.111545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c82657-9340-41a3-bb39-5a7826ea51fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:56.116997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c82657-9340-41a3-bb39-5a7826ea51fc service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:56.124851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1111/2246] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:56.424360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214d032-9d66-40bf-9863-e6775f46f2c6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:56.424820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214d032-9d66-40bf-9863-e6775f46f2c6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:56.424968 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214d032-9d66-40bf-9863-e6775f46f2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:56.469996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1214d032-9d66-40bf-9863-e6775f46f2c6 None None] MFA Rules not processed for user `9348310fd06b473786d1c6be4f20c481`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:56.471289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1214d032-9d66-40bf-9863-e6775f46f2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:56.544199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1136/2247] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:56.563327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c82657-9340-41a3-bb39-5a7826ea51fc service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:56.565126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-81c82657-9340-41a3-bb39-5a7826ea51fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:56.653588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c82657-9340-41a3-bb39-5a7826ea51fc service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:56.659247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277602fe-a4ab-476b-95ed-2845c1d1f5a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:56.659892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277602fe-a4ab-476b-95ed-2845c1d1f5a6 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:56.660173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277602fe-a4ab-476b-95ed-2845c1d1f5a6 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:56.660449 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277602fe-a4ab-476b-95ed-2845c1d1f5a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:56.668846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277602fe-a4ab-476b-95ed-2845c1d1f5a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:56.670631 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277602fe-a4ab-476b-95ed-2845c1d1f5a6 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:56.715522 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1112/2248] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:57.002144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214d032-9d66-40bf-9863-e6775f46f2c6 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:57.004164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1214d032-9d66-40bf-9863-e6775f46f2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:57.018053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214d032-9d66-40bf-9863-e6775f46f2c6 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:57.028797 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:57.030035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:57.032258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:57.032638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] PATH_INFO: `/v3/projects/f2520d8f6bbb42eab71dfe2a2b22180a` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:57.034078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2520d8f6bbb42eab71dfe2a2b22180a)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:57.036520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:57.228095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2520d8f6bbb42eab71dfe2a2b22180a', 'request_id': 'req-df201edc-d462-48b5-a594-5cc43eb0d99f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:57.333843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2520d8f6bbb42eab71dfe2a2b22180a', 'request_id': 'req-df201edc-d462-48b5-a594-5cc43eb0d99f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:57.344105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2520d8f6bbb42eab71dfe2a2b22180a', 'request_id': 'req-df201edc-d462-48b5-a594-5cc43eb0d99f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:57.355573 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] The token cache is being invalidate because project f2520d8f6bbb42eab71dfe2a2b22180a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:57.356227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:57.357100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:57.358705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1137/2249] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:57 2022] DELETE /identity/v3/projects/f2520d8f6bbb42eab71dfe2a2b22180a => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:57.363023 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277602fe-a4ab-476b-95ed-2845c1d1f5a6 service nova] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:57.373676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277602fe-a4ab-476b-95ed-2845c1d1f5a6 service nova] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:57.436280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558b3ed-e07f-4aa1-9d98-3683701b6119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:57.437234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558b3ed-e07f-4aa1-9d98-3683701b6119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:57.437611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558b3ed-e07f-4aa1-9d98-3683701b6119 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:57.437927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558b3ed-e07f-4aa1-9d98-3683701b6119 admin admin] PATH_INFO: `/v3/users/a0e1407cb1864ac691cb87d80448a571` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:57.439498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558b3ed-e07f-4aa1-9d98-3683701b6119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e1407cb1864ac691cb87d80448a571)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:57.441481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558b3ed-e07f-4aa1-9d98-3683701b6119 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:57.661484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8558b3ed-e07f-4aa1-9d98-3683701b6119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e1407cb1864ac691cb87d80448a571'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:57.662775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8558b3ed-e07f-4aa1-9d98-3683701b6119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e1407cb1864ac691cb87d80448a571'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:57.679024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8558b3ed-e07f-4aa1-9d98-3683701b6119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e1407cb1864ac691cb87d80448a571'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:57.850275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8558b3ed-e07f-4aa1-9d98-3683701b6119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e1407cb1864ac691cb87d80448a571'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:57.873244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8558b3ed-e07f-4aa1-9d98-3683701b6119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e1407cb1864ac691cb87d80448a571'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:57.883924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1113/2250] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:51:57 2022] DELETE /identity/v3/users/a0e1407cb1864ac691cb87d80448a571 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:58.518367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:58.518367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:58.559044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df201edc-d462-48b5-a594-5cc43eb0d99f admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:58.661744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:58.668139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:58.668139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:58.668139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] PATH_INFO: `/v3/projects/7fea64ed980e40f8990093f9b717c6dc` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:58.668139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fea64ed980e40f8990093f9b717c6dc)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:58.668139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:58.837719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:58.869832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fea64ed980e40f8990093f9b717c6dc', 'request_id': 'req-11c0318c-3022-4079-99df-473b26fc28c9'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:59.096472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fea64ed980e40f8990093f9b717c6dc', 'request_id': 'req-11c0318c-3022-4079-99df-473b26fc28c9'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:59.104150 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fea64ed980e40f8990093f9b717c6dc', 'request_id': 'req-11c0318c-3022-4079-99df-473b26fc28c9'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:59.123974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] The token cache is being invalidate because project 7fea64ed980e40f8990093f9b717c6dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:51:59.125211 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:59.126057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-11c0318c-3022-4079-99df-473b26fc28c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:51:59.128303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1138/2251] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:51:58 2022] DELETE /identity/v3/projects/7fea64ed980e40f8990093f9b717c6dc => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:51:59.132851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfb2ef-d24d-4776-b304-52a806c9d438 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:59.133537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfb2ef-d24d-4776-b304-52a806c9d438 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:59.133537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfb2ef-d24d-4776-b304-52a806c9d438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:59.155275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-5cbfb2ef-d24d-4776-b304-52a806c9d438 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:59.156600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5cbfb2ef-d24d-4776-b304-52a806c9d438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:59.249022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1114/2252] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:59.254070 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:59.258550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db115123-2d14-4981-92fd-a46ce0349206 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:59.260199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db115123-2d14-4981-92fd-a46ce0349206 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:59.260199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db115123-2d14-4981-92fd-a46ce0349206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:59.281569 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-db115123-2d14-4981-92fd-a46ce0349206 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:59.282825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-db115123-2d14-4981-92fd-a46ce0349206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:59.295500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1139/2253] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:59.297367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfb2ef-d24d-4776-b304-52a806c9d438 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:59.310403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfb2ef-d24d-4776-b304-52a806c9d438 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:59.316789 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cafd5-fc0f-4840-87f4-22df839514a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:59.317693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cafd5-fc0f-4840-87f4-22df839514a0 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:59.320100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cafd5-fc0f-4840-87f4-22df839514a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:59.320388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cafd5-fc0f-4840-87f4-22df839514a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:59.321395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cafd5-fc0f-4840-87f4-22df839514a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:51:59.321714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cafd5-fc0f-4840-87f4-22df839514a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:59.324527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cafd5-fc0f-4840-87f4-22df839514a0 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:59.345227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1115/2254] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:51:59.346140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db115123-2d14-4981-92fd-a46ce0349206 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:59.361970 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db115123-2d14-4981-92fd-a46ce0349206 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:59.368188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fb44a-ad32-4579-bb1c-4a05b5c32e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:59.368914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fb44a-ad32-4579-bb1c-4a05b5c32e8d admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:59.369329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fb44a-ad32-4579-bb1c-4a05b5c32e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:59.369801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fb44a-ad32-4579-bb1c-4a05b5c32e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:59.370602 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fb44a-ad32-4579-bb1c-4a05b5c32e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:59.372732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fb44a-ad32-4579-bb1c-4a05b5c32e8d admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:59.560355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1140/2255] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:51:59 2022] POST /identity/v3/projects => generated 387 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:59.563498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cafd5-fc0f-4840-87f4-22df839514a0 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:59.579782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cafd5-fc0f-4840-87f4-22df839514a0 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:59.700351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ff3b8-0e37-4bc0-a44c-1d839da8c2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:59.701047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff3b8-0e37-4bc0-a44c-1d839da8c2cb admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:59.706102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff3b8-0e37-4bc0-a44c-1d839da8c2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:59.706102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff3b8-0e37-4bc0-a44c-1d839da8c2cb admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:59.706102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ff3b8-0e37-4bc0-a44c-1d839da8c2cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:59.706102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ff3b8-0e37-4bc0-a44c-1d839da8c2cb admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:59.881662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1116/2256] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:51:59 2022] POST /identity/v3/users => generated 294 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:51:59.889034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fb44a-ad32-4579-bb1c-4a05b5c32e8d admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:59.900206 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fb44a-ad32-4579-bb1c-4a05b5c32e8d admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:59.907017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f574a5d-2a24-4aa9-a7dc-43bf57843cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:59.907639 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f574a5d-2a24-4aa9-a7dc-43bf57843cbf admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:59.911690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f574a5d-2a24-4aa9-a7dc-43bf57843cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:59.911690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f574a5d-2a24-4aa9-a7dc-43bf57843cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:59.911690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f574a5d-2a24-4aa9-a7dc-43bf57843cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:59.911690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f574a5d-2a24-4aa9-a7dc-43bf57843cbf admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:59.925846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1141/2257] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:59.929672 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ff3b8-0e37-4bc0-a44c-1d839da8c2cb admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:59.943660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ff3b8-0e37-4bc0-a44c-1d839da8c2cb admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:59.949302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc73ce-a60d-4fdc-864f-6676ad68f11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:59.949934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc73ce-a60d-4fdc-864f-6676ad68f11d admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:59.950220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc73ce-a60d-4fdc-864f-6676ad68f11d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:59.950500 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc73ce-a60d-4fdc-864f-6676ad68f11d admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:59.951163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc73ce-a60d-4fdc-864f-6676ad68f11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:59.953843 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc73ce-a60d-4fdc-864f-6676ad68f11d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:59.968551 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1117/2258] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:59.969824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f574a5d-2a24-4aa9-a7dc-43bf57843cbf admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:59.981005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f574a5d-2a24-4aa9-a7dc-43bf57843cbf admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:59.986058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ecebe-1ac4-44d6-92dc-8b65405a6d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:59.986776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ecebe-1ac4-44d6-92dc-8b65405a6d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:59.988918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ecebe-1ac4-44d6-92dc-8b65405a6d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:59.988918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ecebe-1ac4-44d6-92dc-8b65405a6d61 admin admin] PATH_INFO: `/v3/projects/22557226668e4b4c98847eb6b7548c50/users/c6c534f39cea4a6284b5975ceb5cbaf4/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:00.003565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ecebe-1ac4-44d6-92dc-8b65405a6d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22557226668e4b4c98847eb6b7548c50, user_id=c6c534f39cea4a6284b5975ceb5cbaf4, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:00.006332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ecebe-1ac4-44d6-92dc-8b65405a6d61 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:00.130302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1142/2259] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:51:59 2022] PUT /identity/v3/projects/22557226668e4b4c98847eb6b7548c50/users/c6c534f39cea4a6284b5975ceb5cbaf4/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:00.141346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b776-1d70-4b9f-8e69-e2411f7a68d5 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:00.141889 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b776-1d70-4b9f-8e69-e2411f7a68d5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:00.142113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b776-1d70-4b9f-8e69-e2411f7a68d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:00.173001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-bbf5b776-1d70-4b9f-8e69-e2411f7a68d5 None None] MFA Rules not processed for user `c6c534f39cea4a6284b5975ceb5cbaf4`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:00.174314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-bbf5b776-1d70-4b9f-8e69-e2411f7a68d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:00.263906 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1118/2260] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:52:00.266311 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ecebe-1ac4-44d6-92dc-8b65405a6d61 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:00.340926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ecebe-1ac4-44d6-92dc-8b65405a6d61 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:00.346020 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0a520-6edb-4e01-98fa-38645063bc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:00.346910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0a520-6edb-4e01-98fa-38645063bc6d admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:00.347274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0a520-6edb-4e01-98fa-38645063bc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:00.347667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0a520-6edb-4e01-98fa-38645063bc6d admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:00.348472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0a520-6edb-4e01-98fa-38645063bc6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:00.350625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0a520-6edb-4e01-98fa-38645063bc6d admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:00.550270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced7999-a0f0-4f80-9bc2-b5de748a28a1 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:00.550592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced7999-a0f0-4f80-9bc2-b5de748a28a1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:00.550997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced7999-a0f0-4f80-9bc2-b5de748a28a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:00.574583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-cced7999-a0f0-4f80-9bc2-b5de748a28a1 None None] MFA Rules not processed for user `9348310fd06b473786d1c6be4f20c481`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:00.576138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cced7999-a0f0-4f80-9bc2-b5de748a28a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:00.640290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1119/2261] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:00.641401 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced7999-a0f0-4f80-9bc2-b5de748a28a1 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:00.642828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cced7999-a0f0-4f80-9bc2-b5de748a28a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:00.656729 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced7999-a0f0-4f80-9bc2-b5de748a28a1 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:00.661627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:00.662270 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:00.662564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:00.662838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] PATH_INFO: `/v3/users/f725cad8009144f8b0fd3f3db0d71942` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:00.664070 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f725cad8009144f8b0fd3f3db0d71942)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:00.666100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:00.720500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1143/2262] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:52:00 2022] POST /identity/v3/projects => generated 387 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:00.720500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0a520-6edb-4e01-98fa-38645063bc6d admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:00.720500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-fff0a520-6edb-4e01-98fa-38645063bc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:00.784564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0a520-6edb-4e01-98fa-38645063bc6d admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:00.789736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61581b-8a7f-40f7-8aa5-adde9c8d457a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:00.793849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61581b-8a7f-40f7-8aa5-adde9c8d457a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:00.793849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61581b-8a7f-40f7-8aa5-adde9c8d457a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:00.793849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61581b-8a7f-40f7-8aa5-adde9c8d457a admin admin] PATH_INFO: `/v3/users/f1f9ffb090b340b58a92fca86fbd913e` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:00.793849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61581b-8a7f-40f7-8aa5-adde9c8d457a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1f9ffb090b340b58a92fca86fbd913e)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:00.794211 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61581b-8a7f-40f7-8aa5-adde9c8d457a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:01.005972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f725cad8009144f8b0fd3f3db0d71942'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:01.024741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f725cad8009144f8b0fd3f3db0d71942'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:01.029377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f725cad8009144f8b0fd3f3db0d71942'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:01.135431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f725cad8009144f8b0fd3f3db0d71942'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:01.144511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f725cad8009144f8b0fd3f3db0d71942'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:01.147461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ff61581b-8a7f-40f7-8aa5-adde9c8d457a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1f9ffb090b340b58a92fca86fbd913e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:01.152660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1120/2263] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:00 2022] DELETE /identity/v3/users/f725cad8009144f8b0fd3f3db0d71942 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:01.153945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:01.157605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:01.175857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ff61581b-8a7f-40f7-8aa5-adde9c8d457a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1f9ffb090b340b58a92fca86fbd913e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:01.177248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ff61581b-8a7f-40f7-8aa5-adde9c8d457a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1f9ffb090b340b58a92fca86fbd913e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:01.178380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ff61581b-8a7f-40f7-8aa5-adde9c8d457a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1f9ffb090b340b58a92fca86fbd913e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:01.238142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b209cdf-eac8-4ea7-b74a-8bef4682bdcd admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:01.242750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a2e89-6c20-4b65-a43a-79a006cb6cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:01.243356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a2e89-6c20-4b65-a43a-79a006cb6cad admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:01.243640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a2e89-6c20-4b65-a43a-79a006cb6cad admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:01.244027 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a2e89-6c20-4b65-a43a-79a006cb6cad admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:01.244647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a2e89-6c20-4b65-a43a-79a006cb6cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:01.246564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a2e89-6c20-4b65-a43a-79a006cb6cad admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:01.319213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ff61581b-8a7f-40f7-8aa5-adde9c8d457a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1f9ffb090b340b58a92fca86fbd913e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:01.332592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1144/2264] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:00 2022] DELETE /identity/v3/users/f1f9ffb090b340b58a92fca86fbd913e => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:01.333815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0bdea-8d89-440c-b2f7-75300d84ebad None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:01.334103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0bdea-8d89-440c-b2f7-75300d84ebad None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:01.334378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0bdea-8d89-440c-b2f7-75300d84ebad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:01.353726 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-85e0bdea-8d89-440c-b2f7-75300d84ebad None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:01.354939 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-85e0bdea-8d89-440c-b2f7-75300d84ebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:01.413434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1145/2265] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:01.414695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31590179-0f9e-4e2e-9c75-9a0d7788cca8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:01.414989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31590179-0f9e-4e2e-9c75-9a0d7788cca8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:01.415270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31590179-0f9e-4e2e-9c75-9a0d7788cca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:01.435711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-31590179-0f9e-4e2e-9c75-9a0d7788cca8 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:01.437807 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-31590179-0f9e-4e2e-9c75-9a0d7788cca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:01.442910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1121/2266] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:52:01 2022] POST /identity/v3/users => generated 300 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:01.443766 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a2e89-6c20-4b65-a43a-79a006cb6cad admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:01.445554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8f9a2e89-6c20-4b65-a43a-79a006cb6cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:01.456039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1146/2267] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:01.457052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31590179-0f9e-4e2e-9c75-9a0d7788cca8 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:01.459473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-31590179-0f9e-4e2e-9c75-9a0d7788cca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:01.479430 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31590179-0f9e-4e2e-9c75-9a0d7788cca8 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:01.503589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fcea4-af54-4cc9-aa66-8ef7ad79fa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:01.504722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fcea4-af54-4cc9-aa66-8ef7ad79fa88 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:01.505007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fcea4-af54-4cc9-aa66-8ef7ad79fa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:01.505282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fcea4-af54-4cc9-aa66-8ef7ad79fa88 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:01.505939 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fcea4-af54-4cc9-aa66-8ef7ad79fa88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:01.511706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fcea4-af54-4cc9-aa66-8ef7ad79fa88 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:01.540412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1147/2268] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:01.541458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fcea4-af54-4cc9-aa66-8ef7ad79fa88 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:01.543408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a2e89-6c20-4b65-a43a-79a006cb6cad admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:01.646076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1bf2e-ed84-42ed-87c7-d9add9b2ba5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:01.646700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1bf2e-ed84-42ed-87c7-d9add9b2ba5a service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:01.646982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1bf2e-ed84-42ed-87c7-d9add9b2ba5a service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:01.647261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1bf2e-ed84-42ed-87c7-d9add9b2ba5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:01.657953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fcea4-af54-4cc9-aa66-8ef7ad79fa88 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:01.673317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903dd87-d4e8-47ca-a815-378a84f97bad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:01.675983 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903dd87-d4e8-47ca-a815-378a84f97bad service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:01.676263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903dd87-d4e8-47ca-a815-378a84f97bad service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:01.676538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903dd87-d4e8-47ca-a815-378a84f97bad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:01.676823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1bf2e-ed84-42ed-87c7-d9add9b2ba5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:01.680672 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1bf2e-ed84-42ed-87c7-d9add9b2ba5a service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:01.700744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903dd87-d4e8-47ca-a815-378a84f97bad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:01.702196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903dd87-d4e8-47ca-a815-378a84f97bad service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:01.725380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1122/2269] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:01.726236 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1bf2e-ed84-42ed-87c7-d9add9b2ba5a service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:01.727745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1148/2270] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:01.740090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1bf2e-ed84-42ed-87c7-d9add9b2ba5a service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:01.746679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc963e0-d868-402c-81b2-98427d132376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:01.755772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc963e0-d868-402c-81b2-98427d132376 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:01.755772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc963e0-d868-402c-81b2-98427d132376 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:01.755772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc963e0-d868-402c-81b2-98427d132376 admin admin] PATH_INFO: `/v3/projects/54f8c35f0e2d471ebecee80c3811d4e2/users/69a7c91acd634daaa52c928f046a0e82/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:01.765879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc963e0-d868-402c-81b2-98427d132376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f8c35f0e2d471ebecee80c3811d4e2, user_id=69a7c91acd634daaa52c928f046a0e82, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:01.768848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc963e0-d868-402c-81b2-98427d132376 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:01.922074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1123/2271] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:01 2022] PUT /identity/v3/projects/54f8c35f0e2d471ebecee80c3811d4e2/users/69a7c91acd634daaa52c928f046a0e82/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:01.926328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903dd87-d4e8-47ca-a815-378a84f97bad service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:01.936182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903dd87-d4e8-47ca-a815-378a84f97bad service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:02.003327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80abe25-3f3b-4733-a019-f99671e832bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:02.007308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80abe25-3f3b-4733-a019-f99671e832bc admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:02.007308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80abe25-3f3b-4733-a019-f99671e832bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:02.007308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80abe25-3f3b-4733-a019-f99671e832bc admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:02.007308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80abe25-3f3b-4733-a019-f99671e832bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:02.007308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80abe25-3f3b-4733-a019-f99671e832bc admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:02.026247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1149/2272] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:02.034601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc963e0-d868-402c-81b2-98427d132376 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:02.050606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc963e0-d868-402c-81b2-98427d132376 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:02.055663 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aacc3-5adb-4f08-b87f-0457ded73baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:02.056438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aacc3-5adb-4f08-b87f-0457ded73baf admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:02.056810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aacc3-5adb-4f08-b87f-0457ded73baf admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:02.057159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aacc3-5adb-4f08-b87f-0457ded73baf admin admin] PATH_INFO: `/v3/projects/54f8c35f0e2d471ebecee80c3811d4e2/users/69a7c91acd634daaa52c928f046a0e82/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:02.059545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4aacc3-5adb-4f08-b87f-0457ded73baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f8c35f0e2d471ebecee80c3811d4e2, user_id=69a7c91acd634daaa52c928f046a0e82, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:02.062562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4aacc3-5adb-4f08-b87f-0457ded73baf admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:02.089241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1124/2273] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:02 2022] PUT /identity/v3/projects/54f8c35f0e2d471ebecee80c3811d4e2/users/69a7c91acd634daaa52c928f046a0e82/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:02.099823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7186d-9b68-4c85-a6e5-56a3e6384785 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:02.100229 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7186d-9b68-4c85-a6e5-56a3e6384785 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:02.100537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7186d-9b68-4c85-a6e5-56a3e6384785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:02.155607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-10a7186d-9b68-4c85-a6e5-56a3e6384785 None None] MFA Rules not processed for user `69a7c91acd634daaa52c928f046a0e82`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:02.155607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-10a7186d-9b68-4c85-a6e5-56a3e6384785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:02.157134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:02.306460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1150/2274] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:02.308151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aacc3-5adb-4f08-b87f-0457ded73baf admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:02.311320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ee4aacc3-5adb-4f08-b87f-0457ded73baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:02.381684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7186d-9b68-4c85-a6e5-56a3e6384785 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:02.383682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-10a7186d-9b68-4c85-a6e5-56a3e6384785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:02.383798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aacc3-5adb-4f08-b87f-0457ded73baf admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:02.387774 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181b9ea-715f-4526-9436-52cceabc5a31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:02.388603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181b9ea-715f-4526-9436-52cceabc5a31 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:02.388844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181b9ea-715f-4526-9436-52cceabc5a31 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:02.389288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181b9ea-715f-4526-9436-52cceabc5a31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:02.406917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181b9ea-715f-4526-9436-52cceabc5a31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:02.408441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181b9ea-715f-4526-9436-52cceabc5a31 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:02.429219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1125/2275] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:52:02.465057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7186d-9b68-4c85-a6e5-56a3e6384785 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:02.466307 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181b9ea-715f-4526-9436-52cceabc5a31 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:02.468121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c181b9ea-715f-4526-9436-52cceabc5a31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:02.469160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:02.469944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:02.470319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:02.470712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] PATH_INFO: `/v3/projects/0f8b180b7e7540358026bcdd76488da6` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:02.472032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8b180b7e7540358026bcdd76488da6)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:02.474059 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:02.493003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181b9ea-715f-4526-9436-52cceabc5a31 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:02.511675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:02.512544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:02.513119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:02.513453 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] PATH_INFO: `/v3/projects/7a76c37d3f104207bfd653e68cdc1705` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:02.514684 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a76c37d3f104207bfd653e68cdc1705)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:02.517272 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:02.651298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8b180b7e7540358026bcdd76488da6', 'request_id': 'req-f4e334bd-fadc-45df-afe6-ed775a8c1593'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:02.731212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a76c37d3f104207bfd653e68cdc1705', 'request_id': 'req-848486d8-affe-4a89-99f1-b50726b6ffac'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:02.737704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a76c37d3f104207bfd653e68cdc1705', 'request_id': 'req-848486d8-affe-4a89-99f1-b50726b6ffac'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:02.744383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a76c37d3f104207bfd653e68cdc1705', 'request_id': 'req-848486d8-affe-4a89-99f1-b50726b6ffac'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:02.811154 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8b180b7e7540358026bcdd76488da6', 'request_id': 'req-f4e334bd-fadc-45df-afe6-ed775a8c1593'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:02.819921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8b180b7e7540358026bcdd76488da6', 'request_id': 'req-f4e334bd-fadc-45df-afe6-ed775a8c1593'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:02.831790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] The token cache is being invalidate because project 0f8b180b7e7540358026bcdd76488da6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:02.832130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:02.832863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:02.834636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1151/2276] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:02 2022] DELETE /identity/v3/projects/0f8b180b7e7540358026bcdd76488da6 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:52:02.838484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:02.839822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:02.924976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e334bd-fadc-45df-afe6-ed775a8c1593 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:02.958462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] The token cache is being invalidate because project 7a76c37d3f104207bfd653e68cdc1705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:02.958462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:02.958462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:02.961388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1126/2277] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:02 2022] DELETE /identity/v3/projects/7a76c37d3f104207bfd653e68cdc1705 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:02.962276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:02.963624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:02.985069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:03.001510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:03.002548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:03.002893 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:03.003227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] PATH_INFO: `/v3/users/2588e2fe19a3447e9b33850eba258a17` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:03.005827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2588e2fe19a3447e9b33850eba258a17)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:03.012819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:03.044187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848486d8-affe-4a89-99f1-b50726b6ffac admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:03.049751 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:03.050395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:03.050680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:03.050954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] PATH_INFO: `/v3/users/9bfa8830281e4416b7dc448092d145d8` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:03.052114 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bfa8830281e4416b7dc448092d145d8)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:03.054021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:03.331528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bfa8830281e4416b7dc448092d145d8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:03.331528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bfa8830281e4416b7dc448092d145d8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:03.354452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bfa8830281e4416b7dc448092d145d8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:03.467937 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2588e2fe19a3447e9b33850eba258a17'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:03.480285 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2588e2fe19a3447e9b33850eba258a17'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:03.480986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2588e2fe19a3447e9b33850eba258a17'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:03.481599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2588e2fe19a3447e9b33850eba258a17'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:03.491797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bfa8830281e4416b7dc448092d145d8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:03.517772 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bfa8830281e4416b7dc448092d145d8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:03.526538 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1127/2278] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:02 2022] DELETE /identity/v3/users/9bfa8830281e4416b7dc448092d145d8 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:03.596473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2588e2fe19a3447e9b33850eba258a17'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:03.649698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1152/2279] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:02 2022] DELETE /identity/v3/users/2588e2fe19a3447e9b33850eba258a17 => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:52:04.270254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:04.271752 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:04.333736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:04.335271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:04.370226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcadc0-d825-41ce-8a23-a2dda78180f6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:04.421054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86626b6c-3069-4458-a9be-fa343d19ff49 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:04.426850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:04.427508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:04.427793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:04.428275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] PATH_INFO: `/v3/projects/2e687b6dcfa54cad860531f7599d5ced` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:04.429576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e687b6dcfa54cad860531f7599d5ced)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:04.431610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:04.452229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:04.453921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:04.454576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:04.455144 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] PATH_INFO: `/v3/users/c6c534f39cea4a6284b5975ceb5cbaf4` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:04.457012 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c534f39cea4a6284b5975ceb5cbaf4)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:04.461548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:04.844828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c534f39cea4a6284b5975ceb5cbaf4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:04.845606 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c534f39cea4a6284b5975ceb5cbaf4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:04.855487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e687b6dcfa54cad860531f7599d5ced', 'request_id': 'req-cbd297e1-b804-4963-8a1a-03d72613cec1'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:04.871774 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c534f39cea4a6284b5975ceb5cbaf4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:04.988950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e687b6dcfa54cad860531f7599d5ced', 'request_id': 'req-cbd297e1-b804-4963-8a1a-03d72613cec1'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:04.996040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e687b6dcfa54cad860531f7599d5ced', 'request_id': 'req-cbd297e1-b804-4963-8a1a-03d72613cec1'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.012493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] The token cache is being invalidate because project 2e687b6dcfa54cad860531f7599d5ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:05.012800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.015415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.015415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1153/2280] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:04 2022] DELETE /identity/v3/projects/2e687b6dcfa54cad860531f7599d5ced => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:05.016181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:05.018797 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:05.085786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd297e1-b804-4963-8a1a-03d72613cec1 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:05.090649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:05.091294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:05.091575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:05.091852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] PATH_INFO: `/v3/projects/d731f1ff40a143ed803beb15506163c4` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:05.093047 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d731f1ff40a143ed803beb15506163c4)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:05.095031 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:05.099147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c534f39cea4a6284b5975ceb5cbaf4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.107786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c534f39cea4a6284b5975ceb5cbaf4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.114741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1128/2281] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:04 2022] DELETE /identity/v3/users/c6c534f39cea4a6284b5975ceb5cbaf4 => generated 0 bytes in 845 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:05.118503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:05.119902 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:05.192831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049239f-97cf-4f4f-a86e-397682558a5e admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:05.197602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:05.198266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:05.198554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:05.198833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] PATH_INFO: `/v3/users/b8e4d5bd357f40bf9d54ec7b88602dea` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:05.200088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8e4d5bd357f40bf9d54ec7b88602dea)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:05.202021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:05.286097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd731f1ff40a143ed803beb15506163c4', 'request_id': 'req-50dd6138-9245-4a29-948f-d37b6776a973'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.385761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd731f1ff40a143ed803beb15506163c4', 'request_id': 'req-50dd6138-9245-4a29-948f-d37b6776a973'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.393978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd731f1ff40a143ed803beb15506163c4', 'request_id': 'req-50dd6138-9245-4a29-948f-d37b6776a973'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.405585 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] The token cache is being invalidate because project d731f1ff40a143ed803beb15506163c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:05.406757 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.408088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-50dd6138-9245-4a29-948f-d37b6776a973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.410164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1154/2282] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:05 2022] DELETE /identity/v3/projects/d731f1ff40a143ed803beb15506163c4 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:05.412035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af35eab-33b2-4ebd-9726-adc31f02970b None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:05.412544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af35eab-33b2-4ebd-9726-adc31f02970b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:05.412978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af35eab-33b2-4ebd-9726-adc31f02970b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:05.443258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0af35eab-33b2-4ebd-9726-adc31f02970b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:05.444086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0af35eab-33b2-4ebd-9726-adc31f02970b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:05.513619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1155/2283] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:05.515908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a006e6c-c280-460d-b8ed-58866ac192d5 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:05.515908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a006e6c-c280-460d-b8ed-58866ac192d5 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:05.515908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a006e6c-c280-460d-b8ed-58866ac192d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:05.544244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3a006e6c-c280-460d-b8ed-58866ac192d5 None None] MFA Rules not processed for user `5c01a893d3ab42cc97508768791ff38e`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:05.546136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3a006e6c-c280-460d-b8ed-58866ac192d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:05.547954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8e4d5bd357f40bf9d54ec7b88602dea'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.548854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8e4d5bd357f40bf9d54ec7b88602dea'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.564218 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8e4d5bd357f40bf9d54ec7b88602dea'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.618256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1156/2284] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:05.619414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a006e6c-c280-460d-b8ed-58866ac192d5 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:05.621012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3a006e6c-c280-460d-b8ed-58866ac192d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:05.637082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a006e6c-c280-460d-b8ed-58866ac192d5 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:05.642879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:05.643520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:05.643799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:05.644102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] PATH_INFO: `/v3/users/fa8bef1017f640b786b033a76c952de1` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:05.645293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa8bef1017f640b786b033a76c952de1)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:05.647486 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:05.725794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8e4d5bd357f40bf9d54ec7b88602dea'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.734944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8e4d5bd357f40bf9d54ec7b88602dea'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:05.742066 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1129/2285] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:05 2022] DELETE /identity/v3/users/b8e4d5bd357f40bf9d54ec7b88602dea => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:05.743344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:05.748856 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:05.813929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236b2da-70ba-4ac5-a0d3-ede720811041 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:05.819681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c89e12-d1b8-46b2-9774-457feaa3d7a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:05.828393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c89e12-d1b8-46b2-9774-457feaa3d7a4 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:05.828393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c89e12-d1b8-46b2-9774-457feaa3d7a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:05.828393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c89e12-d1b8-46b2-9774-457feaa3d7a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:05.828393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d2c89e12-d1b8-46b2-9774-457feaa3d7a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:05.919316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c89e12-d1b8-46b2-9774-457feaa3d7a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:05.921071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c89e12-d1b8-46b2-9774-457feaa3d7a4 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:05.941560 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:05.948216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1130/2286] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:06.002672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa8bef1017f640b786b033a76c952de1'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:06.022113 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa8bef1017f640b786b033a76c952de1'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:06.022989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa8bef1017f640b786b033a76c952de1'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:06.023938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa8bef1017f640b786b033a76c952de1'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:06.140105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa8bef1017f640b786b033a76c952de1'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:06.158958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1157/2287] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:05 2022] DELETE /identity/v3/users/fa8bef1017f640b786b033a76c952de1 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:06.746333 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:07.115341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c89e12-d1b8-46b2-9774-457feaa3d7a4 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:07.115341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d2c89e12-d1b8-46b2-9774-457feaa3d7a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:07.180409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:07.182099 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:07.205711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c89e12-d1b8-46b2-9774-457feaa3d7a4 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:07.347060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f4563-baf7-4681-bf93-2ebf7002f5cd admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:07.356278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:07.357233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:07.357753 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:07.358242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] PATH_INFO: `/v3/projects/22557226668e4b4c98847eb6b7548c50` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:07.360235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=22557226668e4b4c98847eb6b7548c50)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:07.362432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:07.380647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:07.384464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:07.384464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:07.384464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] PATH_INFO: `/v3/projects/e62572a469c8487088aade6d12c87daa` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:07.385388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e62572a469c8487088aade6d12c87daa)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:07.387577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:07.513432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22557226668e4b4c98847eb6b7548c50', 'request_id': 'req-ce588a94-be51-48c7-ae8d-08a0626312ca'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:07.612975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22557226668e4b4c98847eb6b7548c50', 'request_id': 'req-ce588a94-be51-48c7-ae8d-08a0626312ca'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:07.619532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22557226668e4b4c98847eb6b7548c50', 'request_id': 'req-ce588a94-be51-48c7-ae8d-08a0626312ca'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:07.631141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e62572a469c8487088aade6d12c87daa', 'request_id': 'req-5fe50b1c-276a-45f5-8509-a0856c40c1e6'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:07.633710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] The token cache is being invalidate because project 22557226668e4b4c98847eb6b7548c50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:07.634018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:07.635129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:07.637653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1158/2288] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:07 2022] DELETE /identity/v3/projects/22557226668e4b4c98847eb6b7548c50 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:07.637905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e62572a469c8487088aade6d12c87daa', 'request_id': 'req-5fe50b1c-276a-45f5-8509-a0856c40c1e6'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:07.641286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:07.642777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:07.645830 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e62572a469c8487088aade6d12c87daa', 'request_id': 'req-5fe50b1c-276a-45f5-8509-a0856c40c1e6'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:07.708167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce588a94-be51-48c7-ae8d-08a0626312ca admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:07.773187 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:07.773935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:07.774223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:07.774501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] PATH_INFO: `/v3/projects/3f35759c931d4afc885632207d8779bf` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:07.776308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f35759c931d4afc885632207d8779bf)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:07.778313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:07.818293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] The token cache is being invalidate because project e62572a469c8487088aade6d12c87daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:07.825495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:07.835476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:07.836117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:07.847812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1131/2289] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:07 2022] DELETE /identity/v3/projects/e62572a469c8487088aade6d12c87daa => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:07.849062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:07.857092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:07.881544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50b1c-276a-45f5-8509-a0856c40c1e6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:07.888759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:07.889526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:07.889889 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:07.890782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] PATH_INFO: `/v3/users/69a7c91acd634daaa52c928f046a0e82` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:07.892148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a7c91acd634daaa52c928f046a0e82)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:07.895397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:07.938714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f35759c931d4afc885632207d8779bf', 'request_id': 'req-b8f89307-c122-4821-9971-e9ee6c84409b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:08.016364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f35759c931d4afc885632207d8779bf', 'request_id': 'req-b8f89307-c122-4821-9971-e9ee6c84409b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:08.027096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f35759c931d4afc885632207d8779bf', 'request_id': 'req-b8f89307-c122-4821-9971-e9ee6c84409b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:08.046656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] The token cache is being invalidate because project 3f35759c931d4afc885632207d8779bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:08.047125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:08.053719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b8f89307-c122-4821-9971-e9ee6c84409b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:08.053719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1159/2290] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:07 2022] DELETE /identity/v3/projects/3f35759c931d4afc885632207d8779bf => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:08.055555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc860738-fdc8-4db0-82a8-2bb923a3648d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:08.056022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc860738-fdc8-4db0-82a8-2bb923a3648d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:08.056495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc860738-fdc8-4db0-82a8-2bb923a3648d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:08.083352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-fc860738-fdc8-4db0-82a8-2bb923a3648d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:08.086337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-fc860738-fdc8-4db0-82a8-2bb923a3648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:08.163683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1160/2291] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:08.165089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cde4e3-603e-4c4f-8b50-0ebd76d77e56 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:08.171181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cde4e3-603e-4c4f-8b50-0ebd76d77e56 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:08.171181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cde4e3-603e-4c4f-8b50-0ebd76d77e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:08.176809 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a7c91acd634daaa52c928f046a0e82'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:08.178030 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a7c91acd634daaa52c928f046a0e82'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:08.188818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-53cde4e3-603e-4c4f-8b50-0ebd76d77e56 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:08.190604 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-53cde4e3-603e-4c4f-8b50-0ebd76d77e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:08.193814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a7c91acd634daaa52c928f046a0e82'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:08.198610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1161/2292] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:08.200377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3adb92-7506-40f9-aaff-4367eb16b2f9 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:08.200766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3adb92-7506-40f9-aaff-4367eb16b2f9 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:08.201139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3adb92-7506-40f9-aaff-4367eb16b2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:08.223830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-da3adb92-7506-40f9-aaff-4367eb16b2f9 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:08.225175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-da3adb92-7506-40f9-aaff-4367eb16b2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:08.236882 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1162/2293] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:08.238147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf5e5c-65c4-4c3b-b0ee-11c7569901dd None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:08.238429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf5e5c-65c4-4c3b-b0ee-11c7569901dd None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:08.238702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf5e5c-65c4-4c3b-b0ee-11c7569901dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:08.262589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1bcf5e5c-65c4-4c3b-b0ee-11c7569901dd None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:08.264336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1bcf5e5c-65c4-4c3b-b0ee-11c7569901dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:08.274612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1163/2294] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:08.275771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf5e5c-65c4-4c3b-b0ee-11c7569901dd None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:08.286612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf5e5c-65c4-4c3b-b0ee-11c7569901dd None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:08.291538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3da5f8-06b5-497a-9613-ce600902880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:08.292529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3da5f8-06b5-497a-9613-ce600902880d admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:08.292954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3da5f8-06b5-497a-9613-ce600902880d admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:08.293360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3da5f8-06b5-497a-9613-ce600902880d admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:08.294202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3da5f8-06b5-497a-9613-ce600902880d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:52:08.294656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3da5f8-06b5-497a-9613-ce600902880d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:08.296577 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3da5f8-06b5-497a-9613-ce600902880d admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:08.296706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a7c91acd634daaa52c928f046a0e82'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:08.310357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1164/2295] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:52:08.317187 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3da5f8-06b5-497a-9613-ce600902880d admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:08.317339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a7c91acd634daaa52c928f046a0e82'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:08.323345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3da5f8-06b5-497a-9613-ce600902880d admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:08.325827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1132/2296] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:07 2022] DELETE /identity/v3/users/69a7c91acd634daaa52c928f046a0e82 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:08.327550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:08.329831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:08.404101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3392a4-d62c-40e9-b53c-8d8b973c1a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:08.404489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3392a4-d62c-40e9-b53c-8d8b973c1a70 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:08.404776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3392a4-d62c-40e9-b53c-8d8b973c1a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:08.405058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3392a4-d62c-40e9-b53c-8d8b973c1a70 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:08.406370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3392a4-d62c-40e9-b53c-8d8b973c1a70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:52:08.407161 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3392a4-d62c-40e9-b53c-8d8b973c1a70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:08.409326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3392a4-d62c-40e9-b53c-8d8b973c1a70 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:08.423490 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1165/2297] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:52:08.428669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3392a4-d62c-40e9-b53c-8d8b973c1a70 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:08.432440 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bf3392a4-d62c-40e9-b53c-8d8b973c1a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:08.444926 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d382d-01ee-478e-a292-34f2f807a1fc admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:08.450824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3392a4-d62c-40e9-b53c-8d8b973c1a70 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:08.457032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354aa66-ee68-4a31-a363-25a11706c044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:08.457594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e920d-0d42-4407-a54e-6d540187ec36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:08.457720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354aa66-ee68-4a31-a363-25a11706c044 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:08.457930 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354aa66-ee68-4a31-a363-25a11706c044 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:08.458216 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354aa66-ee68-4a31-a363-25a11706c044 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:08.458855 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354aa66-ee68-4a31-a363-25a11706c044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:08.459611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e920d-0d42-4407-a54e-6d540187ec36 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:08.460297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e920d-0d42-4407-a54e-6d540187ec36 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:08.460906 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354aa66-ee68-4a31-a363-25a11706c044 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:08.461890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e920d-0d42-4407-a54e-6d540187ec36 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:08.463375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e920d-0d42-4407-a54e-6d540187ec36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:08.465498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e920d-0d42-4407-a54e-6d540187ec36 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:08.477243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:08.606039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1166/2298] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:52:08 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:08.613205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354aa66-ee68-4a31-a363-25a11706c044 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:08.635368 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1133/2299] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:52:08 2022] POST /identity/v3/projects => generated 365 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:08.643421 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e920d-0d42-4407-a54e-6d540187ec36 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:08.751293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354aa66-ee68-4a31-a363-25a11706c044 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:08.772121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e920d-0d42-4407-a54e-6d540187ec36 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:08.777756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf30266-5978-4cc3-985a-3fa3c04959be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:08.778999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30266-5978-4cc3-985a-3fa3c04959be admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:08.779591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30266-5978-4cc3-985a-3fa3c04959be admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:08.780085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30266-5978-4cc3-985a-3fa3c04959be admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:08.780904 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf30266-5978-4cc3-985a-3fa3c04959be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:08.782878 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf30266-5978-4cc3-985a-3fa3c04959be admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:08.817631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4bcb4-5253-421a-a01f-01c2b7a9a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:08.818601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4bcb4-5253-421a-a01f-01c2b7a9a8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:08.818601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4bcb4-5253-421a-a01f-01c2b7a9a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:08.824118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4bcb4-5253-421a-a01f-01c2b7a9a8f2 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:08.824118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4bcb4-5253-421a-a01f-01c2b7a9a8f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:08.824118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4bcb4-5253-421a-a01f-01c2b7a9a8f2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:09.031621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1134/2300] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:52:08 2022] POST /identity/v3/users => generated 283 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:09.033042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf30266-5978-4cc3-985a-3fa3c04959be admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:09.038335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1167/2301] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:52:08 2022] POST /identity/v3/users => generated 289 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:52:09.042398 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4bcb4-5253-421a-a01f-01c2b7a9a8f2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:09.050382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf30266-5978-4cc3-985a-3fa3c04959be admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:09.055271 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4bcb4-5253-421a-a01f-01c2b7a9a8f2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:09.056271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154195b-b806-4693-bfe2-a5aca310238a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:09.057064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154195b-b806-4693-bfe2-a5aca310238a admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.057734 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154195b-b806-4693-bfe2-a5aca310238a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.057848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154195b-b806-4693-bfe2-a5aca310238a admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.058969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154195b-b806-4693-bfe2-a5aca310238a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:09.060435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154195b-b806-4693-bfe2-a5aca310238a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:09.073550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5a050-0010-4fb6-935c-3247bd7f2472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:09.074290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5a050-0010-4fb6-935c-3247bd7f2472 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.074643 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5a050-0010-4fb6-935c-3247bd7f2472 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.074989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5a050-0010-4fb6-935c-3247bd7f2472 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.075718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5a050-0010-4fb6-935c-3247bd7f2472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:09.076355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1135/2302] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:09.081246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154195b-b806-4693-bfe2-a5aca310238a admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:09.082567 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5a050-0010-4fb6-935c-3247bd7f2472 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:09.101659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1168/2303] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:09.101852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154195b-b806-4693-bfe2-a5aca310238a admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:09.103335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5a050-0010-4fb6-935c-3247bd7f2472 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:09.117043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103c959-b904-4ff0-ab0b-ae723827b119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:09.118741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103c959-b904-4ff0-ab0b-ae723827b119 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.118741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103c959-b904-4ff0-ab0b-ae723827b119 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.118741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103c959-b904-4ff0-ab0b-ae723827b119 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.119058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103c959-b904-4ff0-ab0b-ae723827b119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:09.120459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103c959-b904-4ff0-ab0b-ae723827b119 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:09.127524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5a050-0010-4fb6-935c-3247bd7f2472 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:09.132464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c02f3d-5ec7-4688-93a9-a48d1b267c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:09.133200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c02f3d-5ec7-4688-93a9-a48d1b267c3b admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.133200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c02f3d-5ec7-4688-93a9-a48d1b267c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.133406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c02f3d-5ec7-4688-93a9-a48d1b267c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.133972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c02f3d-5ec7-4688-93a9-a48d1b267c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:09.135766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c02f3d-5ec7-4688-93a9-a48d1b267c3b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:09.141321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1136/2304] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:52:09.142117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103c959-b904-4ff0-ab0b-ae723827b119 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:09.153625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103c959-b904-4ff0-ab0b-ae723827b119 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:09.158345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1169/2305] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:09.166500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c02f3d-5ec7-4688-93a9-a48d1b267c3b admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:09.181516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a69ca-308d-4996-af72-f194e5201c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:09.182184 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a69ca-308d-4996-af72-f194e5201c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.182614 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a69ca-308d-4996-af72-f194e5201c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.182904 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a69ca-308d-4996-af72-f194e5201c99 admin admin] PATH_INFO: `/v3/projects/49903397c23442758a8c7a6505332dd3/users/0b106478ec3b4ea18e01b39621140878/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.183820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c02f3d-5ec7-4688-93a9-a48d1b267c3b admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:09.191248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbf481-3195-4968-94a4-20838832083f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:09.192057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbf481-3195-4968-94a4-20838832083f admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.195392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbf481-3195-4968-94a4-20838832083f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.195392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbf481-3195-4968-94a4-20838832083f admin admin] PATH_INFO: `/v3/projects/d19b01b0af6c43feb357799d8ae948e1/users/436adf49f3e24e03971017ea6dd33ac8/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.201408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a69ca-308d-4996-af72-f194e5201c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49903397c23442758a8c7a6505332dd3, user_id=0b106478ec3b4ea18e01b39621140878, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:09.205013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a69ca-308d-4996-af72-f194e5201c99 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:09.210156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bbf481-3195-4968-94a4-20838832083f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19b01b0af6c43feb357799d8ae948e1, user_id=436adf49f3e24e03971017ea6dd33ac8, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:09.212959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bbf481-3195-4968-94a4-20838832083f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:09.282148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1137/2306] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:09 2022] PUT /identity/v3/projects/49903397c23442758a8c7a6505332dd3/users/0b106478ec3b4ea18e01b39621140878/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:09.284085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a69ca-308d-4996-af72-f194e5201c99 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:09.286991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3e9a69ca-308d-4996-af72-f194e5201c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:09.326739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1170/2307] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:09 2022] PUT /identity/v3/projects/d19b01b0af6c43feb357799d8ae948e1/users/436adf49f3e24e03971017ea6dd33ac8/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:09.328918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010a139-2a1f-459f-b2df-014aaf7a9f42 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.330560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010a139-2a1f-459f-b2df-014aaf7a9f42 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.335115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010a139-2a1f-459f-b2df-014aaf7a9f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.366148 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1010a139-2a1f-459f-b2df-014aaf7a9f42 None None] MFA Rules not processed for user `0b106478ec3b4ea18e01b39621140878`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:09.367489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1010a139-2a1f-459f-b2df-014aaf7a9f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:09.382697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a69ca-308d-4996-af72-f194e5201c99 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:09.464479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1171/2308] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:09.467167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d977-155f-4db6-a3bf-815b80ba5c6e None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.467290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d977-155f-4db6-a3bf-815b80ba5c6e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.467662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d977-155f-4db6-a3bf-815b80ba5c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.494727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e411d977-155f-4db6-a3bf-815b80ba5c6e None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:09.506806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e411d977-155f-4db6-a3bf-815b80ba5c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:09.507143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:09.507143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.507756 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.508432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] PATH_INFO: `/v3/projects/54f8c35f0e2d471ebecee80c3811d4e2` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.509919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f8c35f0e2d471ebecee80c3811d4e2)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:09.512218 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:09.600281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1172/2309] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:09.600281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411d977-155f-4db6-a3bf-815b80ba5c6e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:09.604722 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e411d977-155f-4db6-a3bf-815b80ba5c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:09.666649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f8c35f0e2d471ebecee80c3811d4e2', 'request_id': 'req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:09.679805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f8c35f0e2d471ebecee80c3811d4e2', 'request_id': 'req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:09.686485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f8c35f0e2d471ebecee80c3811d4e2', 'request_id': 'req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:09.702464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411d977-155f-4db6-a3bf-815b80ba5c6e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:09.754565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c695e-a04a-42d9-a6d6-217e1c964141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:09.755194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c695e-a04a-42d9-a6d6-217e1c964141 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.755477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c695e-a04a-42d9-a6d6-217e1c964141 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.755754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c695e-a04a-42d9-a6d6-217e1c964141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.771666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:09.815857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c695e-a04a-42d9-a6d6-217e1c964141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:09.817410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c695e-a04a-42d9-a6d6-217e1c964141 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:09.827621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1173/2310] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:09.829089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c695e-a04a-42d9-a6d6-217e1c964141 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:09.830441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-964c695e-a04a-42d9-a6d6-217e1c964141 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:09.895348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c695e-a04a-42d9-a6d6-217e1c964141 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:09.900766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7219c5-8987-47a2-9e25-0d44d1510358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:09.901483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7219c5-8987-47a2-9e25-0d44d1510358 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.901730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7219c5-8987-47a2-9e25-0d44d1510358 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.901987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7219c5-8987-47a2-9e25-0d44d1510358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.930521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] The token cache is being invalidate because project 54f8c35f0e2d471ebecee80c3811d4e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:09.930791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:09.932342 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-dc80cd25-e6a5-4d94-8b3e-7a559aa24bad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:09.938037 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1138/2311] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:09 2022] DELETE /identity/v3/projects/54f8c35f0e2d471ebecee80c3811d4e2 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:09.940134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dc2bb-fcac-49c5-83db-0902a2441f57 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:09.940678 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dc2bb-fcac-49c5-83db-0902a2441f57 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:09.941267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dc2bb-fcac-49c5-83db-0902a2441f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:09.964816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e12dc2bb-fcac-49c5-83db-0902a2441f57 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:09.966229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e12dc2bb-fcac-49c5-83db-0902a2441f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:09.982638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7219c5-8987-47a2-9e25-0d44d1510358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:09.984254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7219c5-8987-47a2-9e25-0d44d1510358 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:09.995251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1174/2312] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:10.018112 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7219c5-8987-47a2-9e25-0d44d1510358 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:10.020048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-da7219c5-8987-47a2-9e25-0d44d1510358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:10.037062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7219c5-8987-47a2-9e25-0d44d1510358 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:10.045071 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:10.049745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:10.049745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:10.049745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] PATH_INFO: `/v3/users/bdba39bea160489db18cb424adc0d71c` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:10.049745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdba39bea160489db18cb424adc0d71c)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:10.050091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:10.068390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1139/2313] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:10.077990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bcd0c-b28d-47cf-aef7-9489e19c1b88 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:10.078181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bcd0c-b28d-47cf-aef7-9489e19c1b88 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:10.078362 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bcd0c-b28d-47cf-aef7-9489e19c1b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:10.098574 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-be2bcd0c-b28d-47cf-aef7-9489e19c1b88 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:10.100008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-be2bcd0c-b28d-47cf-aef7-9489e19c1b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:10.112647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1140/2314] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:10.113735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bcd0c-b28d-47cf-aef7-9489e19c1b88 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:10.123062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bcd0c-b28d-47cf-aef7-9489e19c1b88 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:10.128608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f454bf9-7e68-4ece-aa0c-12e87d444d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:10.129223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f454bf9-7e68-4ece-aa0c-12e87d444d32 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:10.129504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f454bf9-7e68-4ece-aa0c-12e87d444d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:10.130163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f454bf9-7e68-4ece-aa0c-12e87d444d32 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:10.130541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f454bf9-7e68-4ece-aa0c-12e87d444d32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:52:10.130844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f454bf9-7e68-4ece-aa0c-12e87d444d32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:10.132790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f454bf9-7e68-4ece-aa0c-12e87d444d32 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:10.144925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1141/2315] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:52:10.149197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f454bf9-7e68-4ece-aa0c-12e87d444d32 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:10.159058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f454bf9-7e68-4ece-aa0c-12e87d444d32 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:10.164274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d1600-dd7b-4e3f-af08-52b0263528c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:10.164890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d1600-dd7b-4e3f-af08-52b0263528c6 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:10.165768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d1600-dd7b-4e3f-af08-52b0263528c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:10.165768 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d1600-dd7b-4e3f-af08-52b0263528c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:10.166068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d1600-dd7b-4e3f-af08-52b0263528c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:10.168009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d1600-dd7b-4e3f-af08-52b0263528c6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:10.288974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1142/2316] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:52:10 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:52:10.293098 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d1600-dd7b-4e3f-af08-52b0263528c6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:10.295636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdba39bea160489db18cb424adc0d71c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:10.310607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdba39bea160489db18cb424adc0d71c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:10.311412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdba39bea160489db18cb424adc0d71c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:10.312736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdba39bea160489db18cb424adc0d71c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:10.364397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d1600-dd7b-4e3f-af08-52b0263528c6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:10.369746 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabe5f2-3164-4ea0-87dc-aa23a2ea8fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:10.370398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabe5f2-3164-4ea0-87dc-aa23a2ea8fac admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:10.370687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabe5f2-3164-4ea0-87dc-aa23a2ea8fac admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:10.370966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabe5f2-3164-4ea0-87dc-aa23a2ea8fac admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:10.371599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabe5f2-3164-4ea0-87dc-aa23a2ea8fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:10.373623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabe5f2-3164-4ea0-87dc-aa23a2ea8fac admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:10.448156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdba39bea160489db18cb424adc0d71c'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:10.464765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1175/2317] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:10 2022] DELETE /identity/v3/users/bdba39bea160489db18cb424adc0d71c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:52:10.545817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1143/2318] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:52:10 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:52:10.553631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:10.555138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:10.635043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a13c8b-f187-446d-b675-e7d9366f4119 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:10.640136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4b79e-a623-4b09-8b8c-621d323e3f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:10.640790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4b79e-a623-4b09-8b8c-621d323e3f65 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:10.641078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4b79e-a623-4b09-8b8c-621d323e3f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:10.641355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4b79e-a623-4b09-8b8c-621d323e3f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:10.642013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4b79e-a623-4b09-8b8c-621d323e3f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:10.643637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4b79e-a623-4b09-8b8c-621d323e3f65 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:10.657955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1176/2319] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:10.659296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabe5f2-3164-4ea0-87dc-aa23a2ea8fac admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:10.671235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabe5f2-3164-4ea0-87dc-aa23a2ea8fac admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:10.677372 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59e1fd-a6a2-4d50-b393-df6379c6a1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:10.678123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59e1fd-a6a2-4d50-b393-df6379c6a1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:10.678710 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59e1fd-a6a2-4d50-b393-df6379c6a1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:10.678975 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59e1fd-a6a2-4d50-b393-df6379c6a1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:10.679707 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59e1fd-a6a2-4d50-b393-df6379c6a1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:10.681969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59e1fd-a6a2-4d50-b393-df6379c6a1e9 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:10.715622 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1144/2320] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:10.720180 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4b79e-a623-4b09-8b8c-621d323e3f65 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:10.732275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4b79e-a623-4b09-8b8c-621d323e3f65 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:10.735197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:10.741893 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1205e890-5a1c-4a6a-950f-f830c753c619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:10.742685 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205e890-5a1c-4a6a-950f-f830c753c619 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:10.743033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205e890-5a1c-4a6a-950f-f830c753c619 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:10.743552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205e890-5a1c-4a6a-950f-f830c753c619 admin admin] PATH_INFO: `/v3/projects/f27a9c7726f44d8e95a48b18a9c3cf19/users/c0d28b02c18c4dc8a0f33db0e50fbacc/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:10.806077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59e1fd-a6a2-4d50-b393-df6379c6a1e9 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:10.808191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9e59e1fd-a6a2-4d50-b393-df6379c6a1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:10.810564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1205e890-5a1c-4a6a-950f-f830c753c619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f27a9c7726f44d8e95a48b18a9c3cf19, user_id=c0d28b02c18c4dc8a0f33db0e50fbacc, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:10.826480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59e1fd-a6a2-4d50-b393-df6379c6a1e9 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:10.831110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1205e890-5a1c-4a6a-950f-f830c753c619 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:10.837856 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:10.838921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:10.839436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:10.839967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] PATH_INFO: `/v3/projects/7fd56f5ac6a84d2cbe5df806ec6f0848` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:10.842683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fd56f5ac6a84d2cbe5df806ec6f0848)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:10.847155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:10.927970 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1177/2321] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:10 2022] PUT /identity/v3/projects/f27a9c7726f44d8e95a48b18a9c3cf19/users/c0d28b02c18c4dc8a0f33db0e50fbacc/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:10.937399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfb6fa-f912-4bfd-b527-da1bbdf5ee1b None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:10.937955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfb6fa-f912-4bfd-b527-da1bbdf5ee1b None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:10.938236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfb6fa-f912-4bfd-b527-da1bbdf5ee1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:10.969244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-a6bfb6fa-f912-4bfd-b527-da1bbdf5ee1b None None] MFA Rules not processed for user `c0d28b02c18c4dc8a0f33db0e50fbacc`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:10.971210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a6bfb6fa-f912-4bfd-b527-da1bbdf5ee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:10.995847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fd56f5ac6a84d2cbe5df806ec6f0848', 'request_id': 'req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.002724 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fd56f5ac6a84d2cbe5df806ec6f0848', 'request_id': 'req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.009652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fd56f5ac6a84d2cbe5df806ec6f0848', 'request_id': 'req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.042825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1178/2322] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:52:11.054762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfb6fa-f912-4bfd-b527-da1bbdf5ee1b None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:11.056274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a6bfb6fa-f912-4bfd-b527-da1bbdf5ee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:11.104333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] The token cache is being invalidate because project 7fd56f5ac6a84d2cbe5df806ec6f0848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:11.104722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.106977 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.114688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1145/2323] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:10 2022] DELETE /identity/v3/projects/7fd56f5ac6a84d2cbe5df806ec6f0848 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:52:11.115765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:11.118527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:11.136056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfb6fa-f912-4bfd-b527-da1bbdf5ee1b None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:11.199001 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b2edf-9a1d-4d32-be50-8237e079ae8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:11.199646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b2edf-9a1d-4d32-be50-8237e079ae8d service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:11.199965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b2edf-9a1d-4d32-be50-8237e079ae8d service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:11.200250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b2edf-9a1d-4d32-be50-8237e079ae8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:11.202200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-510b2edf-9a1d-4d32-be50-8237e079ae8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:11.210625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7cc35-9a5b-458d-9c5b-a16e7cae2901 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:11.216204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecf5da-6fd9-46bb-8cd9-6e860f8058fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:11.219093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecf5da-6fd9-46bb-8cd9-6e860f8058fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:11.219093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecf5da-6fd9-46bb-8cd9-6e860f8058fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:11.219093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecf5da-6fd9-46bb-8cd9-6e860f8058fe admin admin] PATH_INFO: `/v3/users/9aa89085efdc493099215b8222bd16f0` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:11.221859 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecf5da-6fd9-46bb-8cd9-6e860f8058fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa89085efdc493099215b8222bd16f0)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:11.225460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecf5da-6fd9-46bb-8cd9-6e860f8058fe admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:11.305233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b2edf-9a1d-4d32-be50-8237e079ae8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:11.306714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b2edf-9a1d-4d32-be50-8237e079ae8d service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:11.318033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1179/2324] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:52:11.338013 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b2edf-9a1d-4d32-be50-8237e079ae8d service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:11.339493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-510b2edf-9a1d-4d32-be50-8237e079ae8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:11.353833 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b2edf-9a1d-4d32-be50-8237e079ae8d service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:11.358689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:11.359913 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:11.360199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:11.360475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] PATH_INFO: `/v3/users/c0d28b02c18c4dc8a0f33db0e50fbacc` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:11.361617 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0d28b02c18c4dc8a0f33db0e50fbacc)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:11.363585 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:11.502634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-efecf5da-6fd9-46bb-8cd9-6e860f8058fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa89085efdc493099215b8222bd16f0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.503290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-efecf5da-6fd9-46bb-8cd9-6e860f8058fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa89085efdc493099215b8222bd16f0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.515247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-efecf5da-6fd9-46bb-8cd9-6e860f8058fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa89085efdc493099215b8222bd16f0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.609939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-efecf5da-6fd9-46bb-8cd9-6e860f8058fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa89085efdc493099215b8222bd16f0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.620796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-efecf5da-6fd9-46bb-8cd9-6e860f8058fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa89085efdc493099215b8222bd16f0'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.626642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0d28b02c18c4dc8a0f33db0e50fbacc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.632929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1146/2325] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:11 2022] DELETE /identity/v3/users/9aa89085efdc493099215b8222bd16f0 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:11.647970 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0d28b02c18c4dc8a0f33db0e50fbacc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.653197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0d28b02c18c4dc8a0f33db0e50fbacc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.653197 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0d28b02c18c4dc8a0f33db0e50fbacc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.755053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0d28b02c18c4dc8a0f33db0e50fbacc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:11.771590 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1180/2326] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:11 2022] DELETE /identity/v3/users/c0d28b02c18c4dc8a0f33db0e50fbacc => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:52:11.777676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4c1f-de74-45f1-b04e-16b45699b1f5 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:11.777971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4c1f-de74-45f1-b04e-16b45699b1f5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:11.778255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4c1f-de74-45f1-b04e-16b45699b1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:11.799839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:11.809276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-6abe4c1f-de74-45f1-b04e-16b45699b1f5 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:11.811512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6abe4c1f-de74-45f1-b04e-16b45699b1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:11.923922 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1147/2327] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:11.934457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:11.936331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:11.975784 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe4c1f-de74-45f1-b04e-16b45699b1f5 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:11.977849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6abe4c1f-de74-45f1-b04e-16b45699b1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:12.004074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe4c1f-de74-45f1-b04e-16b45699b1f5 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:12.011943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:12.013387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:12.013387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:12.013387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] PATH_INFO: `/v3/projects/ab0e7db625ac43228178231236d03fd2` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:12.032032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0e7db625ac43228178231236d03fd2)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:12.048721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:12.053207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765d6be-f269-46d5-b568-49e907339810 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:12.080214 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b619cf7-fe06-4560-bdc1-6d522471d8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:12.081471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b619cf7-fe06-4560-bdc1-6d522471d8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:12.082078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b619cf7-fe06-4560-bdc1-6d522471d8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:12.082691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b619cf7-fe06-4560-bdc1-6d522471d8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:12.098933 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b619cf7-fe06-4560-bdc1-6d522471d8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:12.100777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b619cf7-fe06-4560-bdc1-6d522471d8a8 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:12.122188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1181/2328] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:12.287026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0e7db625ac43228178231236d03fd2', 'request_id': 'req-1838821f-551c-4444-80f7-7b3d9ab064a2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:12.309927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0e7db625ac43228178231236d03fd2', 'request_id': 'req-1838821f-551c-4444-80f7-7b3d9ab064a2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:12.320938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0e7db625ac43228178231236d03fd2', 'request_id': 'req-1838821f-551c-4444-80f7-7b3d9ab064a2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:12.376072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:12.477053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] The token cache is being invalidate because project ab0e7db625ac43228178231236d03fd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:12.481687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:12.482880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-1838821f-551c-4444-80f7-7b3d9ab064a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:12.490732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1148/2329] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:11 2022] DELETE /identity/v3/projects/ab0e7db625ac43228178231236d03fd2 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:52:12.495792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e62a2-6356-4cbc-a10b-2f35b71b4245 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:12.496347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e62a2-6356-4cbc-a10b-2f35b71b4245 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:12.496806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e62a2-6356-4cbc-a10b-2f35b71b4245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:12.537495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-108e62a2-6356-4cbc-a10b-2f35b71b4245 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:12.541123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-108e62a2-6356-4cbc-a10b-2f35b71b4245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:12.680414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1182/2330] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:12.691057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57125c76-a351-446c-9ec1-b181ae8ac563 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:12.691427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57125c76-a351-446c-9ec1-b181ae8ac563 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:12.691732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57125c76-a351-446c-9ec1-b181ae8ac563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:12.723996 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-57125c76-a351-446c-9ec1-b181ae8ac563 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:12.725506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-57125c76-a351-446c-9ec1-b181ae8ac563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:12.749310 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1149/2331] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:52:12.753765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e62a2-6356-4cbc-a10b-2f35b71b4245 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:12.768468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e62a2-6356-4cbc-a10b-2f35b71b4245 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:12.778795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daec49-d0ce-4d72-8505-17d9e34411a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:12.778795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daec49-d0ce-4d72-8505-17d9e34411a0 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:12.778795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daec49-d0ce-4d72-8505-17d9e34411a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:12.778795 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daec49-d0ce-4d72-8505-17d9e34411a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:12.779182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58daec49-d0ce-4d72-8505-17d9e34411a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:52:12.779182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58daec49-d0ce-4d72-8505-17d9e34411a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:12.780659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58daec49-d0ce-4d72-8505-17d9e34411a0 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:12.804950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1183/2332] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:52:12.809743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57125c76-a351-446c-9ec1-b181ae8ac563 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:12.822935 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57125c76-a351-446c-9ec1-b181ae8ac563 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:12.843971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c6aaf-278a-475c-842c-33499c5cc61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:12.844885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c6aaf-278a-475c-842c-33499c5cc61f admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:12.845274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c6aaf-278a-475c-842c-33499c5cc61f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:12.845655 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c6aaf-278a-475c-842c-33499c5cc61f admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:12.846562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c6aaf-278a-475c-842c-33499c5cc61f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:12.849041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c6aaf-278a-475c-842c-33499c5cc61f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:13.011834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1150/2333] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:52:12 2022] POST /identity/v3/projects => generated 385 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:13.024138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daec49-d0ce-4d72-8505-17d9e34411a0 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:13.063553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daec49-d0ce-4d72-8505-17d9e34411a0 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:13.157927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c6aaf-278a-475c-842c-33499c5cc61f admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:13.160465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-340c6aaf-278a-475c-842c-33499c5cc61f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:13.221764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee856e4c-3f21-4c95-aa6a-5f87c4e6f4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:13.223773 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee856e4c-3f21-4c95-aa6a-5f87c4e6f4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:13.224385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee856e4c-3f21-4c95-aa6a-5f87c4e6f4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:13.225511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee856e4c-3f21-4c95-aa6a-5f87c4e6f4a0 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:13.226307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee856e4c-3f21-4c95-aa6a-5f87c4e6f4a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:13.234275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee856e4c-3f21-4c95-aa6a-5f87c4e6f4a0 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:13.321063 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c6aaf-278a-475c-842c-33499c5cc61f admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:13.335298 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:13.336020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:13.336311 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:13.336590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] PATH_INFO: `/v3/projects/f27a9c7726f44d8e95a48b18a9c3cf19` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:13.337913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f27a9c7726f44d8e95a48b18a9c3cf19)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:13.340030 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:13.363182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1184/2334] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:52:13 2022] POST /identity/v3/users => generated 293 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:13.364331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee856e4c-3f21-4c95-aa6a-5f87c4e6f4a0 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:13.376743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee856e4c-3f21-4c95-aa6a-5f87c4e6f4a0 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:13.381713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a6623-d472-408f-a45b-51fa923cce74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:13.382328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a6623-d472-408f-a45b-51fa923cce74 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:13.382615 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a6623-d472-408f-a45b-51fa923cce74 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:13.382895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a6623-d472-408f-a45b-51fa923cce74 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:13.383554 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a6623-d472-408f-a45b-51fa923cce74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:13.385391 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a6623-d472-408f-a45b-51fa923cce74 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:13.397305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1185/2335] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:13.401316 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a6623-d472-408f-a45b-51fa923cce74 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:13.411971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a6623-d472-408f-a45b-51fa923cce74 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:13.416716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97ad06-17a1-41f3-9a8a-e5532707c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:13.417314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97ad06-17a1-41f3-9a8a-e5532707c07c admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:13.417605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97ad06-17a1-41f3-9a8a-e5532707c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:13.417889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97ad06-17a1-41f3-9a8a-e5532707c07c admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:13.418599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97ad06-17a1-41f3-9a8a-e5532707c07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:13.420232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97ad06-17a1-41f3-9a8a-e5532707c07c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:13.435854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1186/2336] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:52:13.437900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97ad06-17a1-41f3-9a8a-e5532707c07c admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:13.470954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97ad06-17a1-41f3-9a8a-e5532707c07c admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:13.483860 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f27a9c7726f44d8e95a48b18a9c3cf19', 'request_id': 'req-688a3c8f-4385-422d-89c0-c851875c1336'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:13.500923 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f27a9c7726f44d8e95a48b18a9c3cf19', 'request_id': 'req-688a3c8f-4385-422d-89c0-c851875c1336'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:13.514939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f27a9c7726f44d8e95a48b18a9c3cf19', 'request_id': 'req-688a3c8f-4385-422d-89c0-c851875c1336'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:13.599759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1461caa-23e6-402b-a0d2-af3776f37a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:13.601024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1461caa-23e6-402b-a0d2-af3776f37a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:13.601472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1461caa-23e6-402b-a0d2-af3776f37a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:13.601901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1461caa-23e6-402b-a0d2-af3776f37a2e admin admin] PATH_INFO: `/v3/projects/dd54c4fe43304660b0be26c030016087/users/78b439283f844e8296f0bd956c93fa81/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:13.627361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1461caa-23e6-402b-a0d2-af3776f37a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd54c4fe43304660b0be26c030016087, user_id=78b439283f844e8296f0bd956c93fa81, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:13.628040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] The token cache is being invalidate because project f27a9c7726f44d8e95a48b18a9c3cf19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:13.628571 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:13.629932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-688a3c8f-4385-422d-89c0-c851875c1336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:13.631899 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1151/2337] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:13 2022] DELETE /identity/v3/projects/f27a9c7726f44d8e95a48b18a9c3cf19 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:13.635770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1461caa-23e6-402b-a0d2-af3776f37a2e admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:13.638193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f198e7c-2978-449b-8ae2-e01f65d14b57 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:13.638649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f198e7c-2978-449b-8ae2-e01f65d14b57 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:13.639356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f198e7c-2978-449b-8ae2-e01f65d14b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:13.676008 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7f198e7c-2978-449b-8ae2-e01f65d14b57 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:13.677422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7f198e7c-2978-449b-8ae2-e01f65d14b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:13.690611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1152/2338] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:13.699827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d4287-257f-4bd9-90f4-d2b9153c041b None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:13.701106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d4287-257f-4bd9-90f4-d2b9153c041b None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:13.702278 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d4287-257f-4bd9-90f4-d2b9153c041b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:13.759750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-3b8d4287-257f-4bd9-90f4-d2b9153c041b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:13.765326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1187/2339] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:13 2022] PUT /identity/v3/projects/dd54c4fe43304660b0be26c030016087/users/78b439283f844e8296f0bd956c93fa81/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:13.766640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3b8d4287-257f-4bd9-90f4-d2b9153c041b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:13.774895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf1c1b-8f67-4041-abd9-ad787fa3eefb None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:13.775506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf1c1b-8f67-4041-abd9-ad787fa3eefb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:13.775972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf1c1b-8f67-4041-abd9-ad787fa3eefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:13.857421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3daf1c1b-8f67-4041-abd9-ad787fa3eefb None None] MFA Rules not processed for user `78b439283f844e8296f0bd956c93fa81`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:13.858789 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3daf1c1b-8f67-4041-abd9-ad787fa3eefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:13.948928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1153/2340] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:13.950187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d4287-257f-4bd9-90f4-d2b9153c041b None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:13.970636 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d4287-257f-4bd9-90f4-d2b9153c041b None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:13.984617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1135c-4609-4ad7-acd0-99f3a40b50e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:13.985617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1135c-4609-4ad7-acd0-99f3a40b50e2 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:13.986507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1135c-4609-4ad7-acd0-99f3a40b50e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:13.987403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1135c-4609-4ad7-acd0-99f3a40b50e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:13.989951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1135c-4609-4ad7-acd0-99f3a40b50e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:52:13.991518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1135c-4609-4ad7-acd0-99f3a40b50e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:13.994392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1135c-4609-4ad7-acd0-99f3a40b50e2 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:14.038410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1188/2342] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:14.038944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1154/2342] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:52:14.039413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1135c-4609-4ad7-acd0-99f3a40b50e2 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:14.040534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf1c1b-8f67-4041-abd9-ad787fa3eefb None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:14.043932 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3daf1c1b-8f67-4041-abd9-ad787fa3eefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:14.069309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1135c-4609-4ad7-acd0-99f3a40b50e2 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:14.081461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23f2ea-3f46-42d1-9d12-3a451dcd9fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:14.083165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23f2ea-3f46-42d1-9d12-3a451dcd9fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:14.085157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23f2ea-3f46-42d1-9d12-3a451dcd9fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:14.085157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23f2ea-3f46-42d1-9d12-3a451dcd9fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:14.085521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf1c1b-8f67-4041-abd9-ad787fa3eefb None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:14.086714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23f2ea-3f46-42d1-9d12-3a451dcd9fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:14.090573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23f2ea-3f46-42d1-9d12-3a451dcd9fd9 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:14.101680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a544e-c47a-4e54-b9e1-cc53c0656446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:14.101680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a544e-c47a-4e54-b9e1-cc53c0656446 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:14.101680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a544e-c47a-4e54-b9e1-cc53c0656446 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:14.103292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a544e-c47a-4e54-b9e1-cc53c0656446 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:14.105378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a544e-c47a-4e54-b9e1-cc53c0656446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:14.109763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a544e-c47a-4e54-b9e1-cc53c0656446 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:14.249972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1189/2343] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:52:14 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:14.250329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1155/2344] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:52:14 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:14.253703 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23f2ea-3f46-42d1-9d12-3a451dcd9fd9 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:14.255319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a544e-c47a-4e54-b9e1-cc53c0656446 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:14.303616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:14.419310 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23f2ea-3f46-42d1-9d12-3a451dcd9fd9 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:14.438293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbadabf-7e6a-463e-b0e7-0b71440fe0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:14.442410 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbadabf-7e6a-463e-b0e7-0b71440fe0ff admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:14.444465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbadabf-7e6a-463e-b0e7-0b71440fe0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:14.444875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbadabf-7e6a-463e-b0e7-0b71440fe0ff admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:14.445953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbadabf-7e6a-463e-b0e7-0b71440fe0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:14.451265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbadabf-7e6a-463e-b0e7-0b71440fe0ff admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:14.491119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a544e-c47a-4e54-b9e1-cc53c0656446 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:14.497405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e95d0-d114-49bb-8618-490457ebd6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:14.498165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e95d0-d114-49bb-8618-490457ebd6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:14.498469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e95d0-d114-49bb-8618-490457ebd6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:14.498954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e95d0-d114-49bb-8618-490457ebd6f4 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:14.499689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e95d0-d114-49bb-8618-490457ebd6f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:14.502012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e95d0-d114-49bb-8618-490457ebd6f4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:14.627385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1156/2345] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:52:14 2022] POST /identity/v3/users => generated 292 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:14.631623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1190/2346] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:52:14 2022] POST /identity/v3/users => generated 299 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:52:14.632290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbadabf-7e6a-463e-b0e7-0b71440fe0ff admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:14.633079 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e95d0-d114-49bb-8618-490457ebd6f4 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:14.655633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbadabf-7e6a-463e-b0e7-0b71440fe0ff admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:14.661509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e95d0-d114-49bb-8618-490457ebd6f4 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:14.667459 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8e7c8-9f57-42b1-811f-d08bd1c4fb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:14.668139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8e7c8-9f57-42b1-811f-d08bd1c4fb2b admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:14.668425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8e7c8-9f57-42b1-811f-d08bd1c4fb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:14.668703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8e7c8-9f57-42b1-811f-d08bd1c4fb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:14.669376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8e7c8-9f57-42b1-811f-d08bd1c4fb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:14.671226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8e7c8-9f57-42b1-811f-d08bd1c4fb2b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:14.672932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5f8d2-3029-4c41-aba9-8cd4acecdcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:14.673747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5f8d2-3029-4c41-aba9-8cd4acecdcc8 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:14.674168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5f8d2-3029-4c41-aba9-8cd4acecdcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:14.675941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5f8d2-3029-4c41-aba9-8cd4acecdcc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:14.676778 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5f8d2-3029-4c41-aba9-8cd4acecdcc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:14.680976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5f8d2-3029-4c41-aba9-8cd4acecdcc8 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:14.689521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1191/2347] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:14.699950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8e7c8-9f57-42b1-811f-d08bd1c4fb2b admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:14.710074 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1157/2348] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:14.713716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8e7c8-9f57-42b1-811f-d08bd1c4fb2b admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:14.715139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5f8d2-3029-4c41-aba9-8cd4acecdcc8 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:14.720081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb33a4c-ea2c-4f6b-a4fb-f6d75118021c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:14.721026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb33a4c-ea2c-4f6b-a4fb-f6d75118021c admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:14.721387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb33a4c-ea2c-4f6b-a4fb-f6d75118021c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:14.721669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb33a4c-ea2c-4f6b-a4fb-f6d75118021c admin admin] PATH_INFO: `/v3/projects/67c84744199948c98446becfb99dd816/users/1b3ee4c54bcb4bdfa3ca003de4a5b10a/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:14.735415 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5f8d2-3029-4c41-aba9-8cd4acecdcc8 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:14.743063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb33a4c-ea2c-4f6b-a4fb-f6d75118021c admin admin] RBAC: Authorizing `identity:create_grant(project_id=67c84744199948c98446becfb99dd816, user_id=1b3ee4c54bcb4bdfa3ca003de4a5b10a, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:14.746218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb33a4c-ea2c-4f6b-a4fb-f6d75118021c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:14.748497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b84be-c7f7-4472-846a-cd6fe4dfcdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:14.749128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b84be-c7f7-4472-846a-cd6fe4dfcdce admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:14.749414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b84be-c7f7-4472-846a-cd6fe4dfcdce admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:14.749844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b84be-c7f7-4472-846a-cd6fe4dfcdce admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:14.750447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b84be-c7f7-4472-846a-cd6fe4dfcdce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:14.752115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b84be-c7f7-4472-846a-cd6fe4dfcdce admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:14.768137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1158/2349] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:14.769658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b84be-c7f7-4472-846a-cd6fe4dfcdce admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:14.784339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b84be-c7f7-4472-846a-cd6fe4dfcdce admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:14.791047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f2d9f-ee09-4212-a704-320b85f2e6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:14.792412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f2d9f-ee09-4212-a704-320b85f2e6bf admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:14.792927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f2d9f-ee09-4212-a704-320b85f2e6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:14.793489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f2d9f-ee09-4212-a704-320b85f2e6bf admin admin] PATH_INFO: `/v3/projects/6bd35af5756e4a259f1da42a96541ef7/users/a1982f2b840545f29478fa49069c5d2e/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:14.813745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f2d9f-ee09-4212-a704-320b85f2e6bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd35af5756e4a259f1da42a96541ef7, user_id=a1982f2b840545f29478fa49069c5d2e, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:14.818865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f2d9f-ee09-4212-a704-320b85f2e6bf admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:14.893276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1192/2350] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:14 2022] PUT /identity/v3/projects/67c84744199948c98446becfb99dd816/users/1b3ee4c54bcb4bdfa3ca003de4a5b10a/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:14.900598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb33a4c-ea2c-4f6b-a4fb-f6d75118021c admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:15.086211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1159/2351] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:14 2022] PUT /identity/v3/projects/6bd35af5756e4a259f1da42a96541ef7/users/a1982f2b840545f29478fa49069c5d2e/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:15.101916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8243962-1d17-4323-a56a-9027b07ddbe3 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:15.102271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8243962-1d17-4323-a56a-9027b07ddbe3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:15.102579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8243962-1d17-4323-a56a-9027b07ddbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:15.118955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb33a4c-ea2c-4f6b-a4fb-f6d75118021c admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:15.150565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:15.172928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-f8243962-1d17-4323-a56a-9027b07ddbe3 None None] MFA Rules not processed for user `a1982f2b840545f29478fa49069c5d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:15.174206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f8243962-1d17-4323-a56a-9027b07ddbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:15.290408 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed2472-fb0b-4d31-b68a-30b4d871a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:15.291049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed2472-fb0b-4d31-b68a-30b4d871a553 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:15.291345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed2472-fb0b-4d31-b68a-30b4d871a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:15.291625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed2472-fb0b-4d31-b68a-30b4d871a553 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:15.292365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed2472-fb0b-4d31-b68a-30b4d871a553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:15.294033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed2472-fb0b-4d31-b68a-30b4d871a553 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:15.326115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1193/2352] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:52:15.331796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed2472-fb0b-4d31-b68a-30b4d871a553 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:15.350820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed2472-fb0b-4d31-b68a-30b4d871a553 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:15.351690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1160/2353] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:15.362625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7abf0c-16af-43c7-961f-900405bdc461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:15.363393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7abf0c-16af-43c7-961f-900405bdc461 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:15.363683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7abf0c-16af-43c7-961f-900405bdc461 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:15.363997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7abf0c-16af-43c7-961f-900405bdc461 admin admin] PATH_INFO: `/v3/projects/67c84744199948c98446becfb99dd816/users/1b3ee4c54bcb4bdfa3ca003de4a5b10a/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:15.366575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7abf0c-16af-43c7-961f-900405bdc461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67c84744199948c98446becfb99dd816, user_id=1b3ee4c54bcb4bdfa3ca003de4a5b10a, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:15.367398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8243962-1d17-4323-a56a-9027b07ddbe3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:15.369242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f8243962-1d17-4323-a56a-9027b07ddbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:15.371324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7abf0c-16af-43c7-961f-900405bdc461 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:15.388097 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1194/2354] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:15 2022] PUT /identity/v3/projects/67c84744199948c98446becfb99dd816/users/1b3ee4c54bcb4bdfa3ca003de4a5b10a/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:15.395972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2d090-0a38-441e-9df9-7885ee291d16 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:15.396700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2d090-0a38-441e-9df9-7885ee291d16 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:15.397880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2d090-0a38-441e-9df9-7885ee291d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:15.460935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-6ae2d090-0a38-441e-9df9-7885ee291d16 None None] MFA Rules not processed for user `1b3ee4c54bcb4bdfa3ca003de4a5b10a`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:15.464427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6ae2d090-0a38-441e-9df9-7885ee291d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:15.619909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1195/2355] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:52:15.630003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2d090-0a38-441e-9df9-7885ee291d16 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:15.632233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6ae2d090-0a38-441e-9df9-7885ee291d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:15.638879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8243962-1d17-4323-a56a-9027b07ddbe3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:15.703395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7b4fe-0d57-4673-a8de-176d53531874 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:15.704322 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7b4fe-0d57-4673-a8de-176d53531874 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:15.704754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7b4fe-0d57-4673-a8de-176d53531874 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:15.705300 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7b4fe-0d57-4673-a8de-176d53531874 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:15.715632 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2d090-0a38-441e-9df9-7885ee291d16 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:15.721361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a673a26-1034-4b27-b6c6-a7ad4fa75630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:15.722432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a673a26-1034-4b27-b6c6-a7ad4fa75630 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:15.723216 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a673a26-1034-4b27-b6c6-a7ad4fa75630 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:15.723657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a673a26-1034-4b27-b6c6-a7ad4fa75630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:15.804538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a673a26-1034-4b27-b6c6-a7ad4fa75630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:15.806436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a673a26-1034-4b27-b6c6-a7ad4fa75630 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:15.831570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1196/2356] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:15.840105 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7b4fe-0d57-4673-a8de-176d53531874 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:15.846130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7b4fe-0d57-4673-a8de-176d53531874 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:15.870002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1161/2357] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:52:18.042327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:19.166494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:20.449816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fe716-6482-4c0e-babb-e99bd0b5a77f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:20.450499 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fe716-6482-4c0e-babb-e99bd0b5a77f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:20.450499 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fe716-6482-4c0e-babb-e99bd0b5a77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:20.483254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-685fe716-6482-4c0e-babb-e99bd0b5a77f None None] MFA Rules not processed for user `436adf49f3e24e03971017ea6dd33ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:20.484501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-685fe716-6482-4c0e-babb-e99bd0b5a77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:20.551954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1197/2358] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:20.560609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7b4fe-0d57-4673-a8de-176d53531874 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:20.562325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d2e7b4fe-0d57-4673-a8de-176d53531874 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:20.638350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7b4fe-0d57-4673-a8de-176d53531874 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:20.642889 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f17ec7-2b56-485b-ae0c-e75ea0393721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:20.643576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f17ec7-2b56-485b-ae0c-e75ea0393721 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:20.643896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f17ec7-2b56-485b-ae0c-e75ea0393721 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:20.644232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f17ec7-2b56-485b-ae0c-e75ea0393721 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:20.644903 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f17ec7-2b56-485b-ae0c-e75ea0393721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:20.646805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f17ec7-2b56-485b-ae0c-e75ea0393721 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:20.905749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1162/2359] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:52:20 2022] POST /identity/v3/projects => generated 377 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:20.906922 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fe716-6482-4c0e-babb-e99bd0b5a77f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:20.925941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fe716-6482-4c0e-babb-e99bd0b5a77f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:21.010996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1e7d7-2217-4729-99c3-7dd0b2467ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:21.011733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1e7d7-2217-4729-99c3-7dd0b2467ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:21.012138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1e7d7-2217-4729-99c3-7dd0b2467ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:21.012518 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1e7d7-2217-4729-99c3-7dd0b2467ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:21.013267 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1e7d7-2217-4729-99c3-7dd0b2467ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:21.015901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1e7d7-2217-4729-99c3-7dd0b2467ae0 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:21.146944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1198/2360] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:52:20 2022] POST /identity/v3/users => generated 295 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:21.154759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f17ec7-2b56-485b-ae0c-e75ea0393721 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:21.162133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f17ec7-2b56-485b-ae0c-e75ea0393721 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:21.168182 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d8f60-e833-439c-99ec-7e51d44a7bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:21.169068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d8f60-e833-439c-99ec-7e51d44a7bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:21.169359 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d8f60-e833-439c-99ec-7e51d44a7bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:21.169644 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d8f60-e833-439c-99ec-7e51d44a7bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:21.170306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d8f60-e833-439c-99ec-7e51d44a7bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:21.173059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d8f60-e833-439c-99ec-7e51d44a7bf4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:21.186019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1163/2361] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:52:21.190297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1e7d7-2217-4729-99c3-7dd0b2467ae0 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:21.201052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1e7d7-2217-4729-99c3-7dd0b2467ae0 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:21.206653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7527e-1a56-4e77-9a23-c91d392c88a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:21.207421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7527e-1a56-4e77-9a23-c91d392c88a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:21.207783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7527e-1a56-4e77-9a23-c91d392c88a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:21.208179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7527e-1a56-4e77-9a23-c91d392c88a1 admin admin] PATH_INFO: `/v3/projects/ef6bcceb1f214427bbd4a07c5aceb0ed/users/99e270149d4a42849cd73dd1267f15c0/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:21.238718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7527e-1a56-4e77-9a23-c91d392c88a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6bcceb1f214427bbd4a07c5aceb0ed, user_id=99e270149d4a42849cd73dd1267f15c0, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:21.242170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7527e-1a56-4e77-9a23-c91d392c88a1 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:21.354740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1199/2362] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:21 2022] PUT /identity/v3/projects/ef6bcceb1f214427bbd4a07c5aceb0ed/users/99e270149d4a42849cd73dd1267f15c0/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:21.358354 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d8f60-e833-439c-99ec-7e51d44a7bf4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:21.380804 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d8f60-e833-439c-99ec-7e51d44a7bf4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:21.491364 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fbebc-ea08-413a-a20d-47a5881c4de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:21.492214 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fbebc-ea08-413a-a20d-47a5881c4de3 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:21.493043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fbebc-ea08-413a-a20d-47a5881c4de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:21.493327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fbebc-ea08-413a-a20d-47a5881c4de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:21.494094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fbebc-ea08-413a-a20d-47a5881c4de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:21.495722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fbebc-ea08-413a-a20d-47a5881c4de3 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:21.516267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1164/2363] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:52:21.518983 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7527e-1a56-4e77-9a23-c91d392c88a1 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:21.534009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7527e-1a56-4e77-9a23-c91d392c88a1 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:21.539990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fc817-3369-4c87-8b3a-676122d98b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:21.540959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fc817-3369-4c87-8b3a-676122d98b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:21.541428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fc817-3369-4c87-8b3a-676122d98b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:21.542181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fc817-3369-4c87-8b3a-676122d98b2f admin admin] PATH_INFO: `/v3/projects/ef6bcceb1f214427bbd4a07c5aceb0ed/users/99e270149d4a42849cd73dd1267f15c0/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:21.544488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fc817-3369-4c87-8b3a-676122d98b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6bcceb1f214427bbd4a07c5aceb0ed, user_id=99e270149d4a42849cd73dd1267f15c0, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:21.549379 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fc817-3369-4c87-8b3a-676122d98b2f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:21.565281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1200/2364] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:21 2022] PUT /identity/v3/projects/ef6bcceb1f214427bbd4a07c5aceb0ed/users/99e270149d4a42849cd73dd1267f15c0/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:22.541514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:23.141074 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:26.075786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:26.497134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fbebc-ea08-413a-a20d-47a5881c4de3 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:26.562255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fbebc-ea08-413a-a20d-47a5881c4de3 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:26.617215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acbd37-fc17-47e4-9e26-d1bb2a604a04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:26.617839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acbd37-fc17-47e4-9e26-d1bb2a604a04 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:26.618463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acbd37-fc17-47e4-9e26-d1bb2a604a04 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:26.618596 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acbd37-fc17-47e4-9e26-d1bb2a604a04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:26.726162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acbd37-fc17-47e4-9e26-d1bb2a604a04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:26.728437 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acbd37-fc17-47e4-9e26-d1bb2a604a04 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:26.742696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1165/2365] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:26.920731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:28.855204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a15b8-e235-499a-9d55-3ac0c947d823 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:28.856300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a15b8-e235-499a-9d55-3ac0c947d823 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:28.856787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a15b8-e235-499a-9d55-3ac0c947d823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:28.891797 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9e5a15b8-e235-499a-9d55-3ac0c947d823 None None] MFA Rules not processed for user `99e270149d4a42849cd73dd1267f15c0`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:28.893541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9e5a15b8-e235-499a-9d55-3ac0c947d823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:28.980238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1201/2366] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:28.990117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acbd37-fc17-47e4-9e26-d1bb2a604a04 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:28.993104 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-53acbd37-fc17-47e4-9e26-d1bb2a604a04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:29.073873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acbd37-fc17-47e4-9e26-d1bb2a604a04 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:29.086160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1746c58-1738-418d-9430-b382cde5bb0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:29.086160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1746c58-1738-418d-9430-b382cde5bb0e service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:29.086160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1746c58-1738-418d-9430-b382cde5bb0e service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:29.086160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1746c58-1738-418d-9430-b382cde5bb0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:29.110175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:29.167557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1746c58-1738-418d-9430-b382cde5bb0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:29.169215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1746c58-1738-418d-9430-b382cde5bb0e service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:29.180411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1166/2367] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:29.481070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:31.585164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:31.905818 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:32.026676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a15b8-e235-499a-9d55-3ac0c947d823 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:32.115894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a15b8-e235-499a-9d55-3ac0c947d823 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:32.120596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:32.121351 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:32.121625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:32.121907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] PATH_INFO: `/v3/users/a1982f2b840545f29478fa49069c5d2e` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:32.123085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1982f2b840545f29478fa49069c5d2e)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:32.125172 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:32.527912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1982f2b840545f29478fa49069c5d2e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:32.543363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1982f2b840545f29478fa49069c5d2e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:32.544505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1982f2b840545f29478fa49069c5d2e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:32.548650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1982f2b840545f29478fa49069c5d2e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:32.651649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1982f2b840545f29478fa49069c5d2e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:32.668317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1202/2368] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:32 2022] DELETE /identity/v3/users/a1982f2b840545f29478fa49069c5d2e => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:32.672857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a2bb-6092-445b-a3c4-fe921b54d10b None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:32.673262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a2bb-6092-445b-a3c4-fe921b54d10b None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:32.673621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a2bb-6092-445b-a3c4-fe921b54d10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:32.692385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-6982a2bb-6092-445b-a3c4-fe921b54d10b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:32.693593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6982a2bb-6092-445b-a3c4-fe921b54d10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:32.753822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1167/2369] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:32.766289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:32.767976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:32.838793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e18ad7-0788-4a56-8750-6232959668be admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:32.842523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68236-5e02-41fb-adc2-2186c8c425d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:32.843152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68236-5e02-41fb-adc2-2186c8c425d3 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:32.843462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68236-5e02-41fb-adc2-2186c8c425d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:32.844335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68236-5e02-41fb-adc2-2186c8c425d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:32.854281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b68236-5e02-41fb-adc2-2186c8c425d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:32.855736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b68236-5e02-41fb-adc2-2186c8c425d3 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:32.866096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1203/2370] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:33.338981 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982a2bb-6092-445b-a3c4-fe921b54d10b None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:33.339519 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6982a2bb-6092-445b-a3c4-fe921b54d10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:33.354838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982a2bb-6092-445b-a3c4-fe921b54d10b None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:33.360499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:33.361424 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:33.361708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:33.361985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] PATH_INFO: `/v3/projects/6bd35af5756e4a259f1da42a96541ef7` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:33.363151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd35af5756e4a259f1da42a96541ef7)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:33.365640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:33.517095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd35af5756e4a259f1da42a96541ef7', 'request_id': 'req-8765fff8-4294-440a-b9fc-98dc7f8add9b'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:33.524025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd35af5756e4a259f1da42a96541ef7', 'request_id': 'req-8765fff8-4294-440a-b9fc-98dc7f8add9b'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:33.535379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd35af5756e4a259f1da42a96541ef7', 'request_id': 'req-8765fff8-4294-440a-b9fc-98dc7f8add9b'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:33.680287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] The token cache is being invalidate because project 6bd35af5756e4a259f1da42a96541ef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:33.680287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:33.680287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8765fff8-4294-440a-b9fc-98dc7f8add9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:33.680287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1168/2371] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:33 2022] DELETE /identity/v3/projects/6bd35af5756e4a259f1da42a96541ef7 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:33.683599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0580d-12a8-4023-9c13-9da55a548d31 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:33.684003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0580d-12a8-4023-9c13-9da55a548d31 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:33.684201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0580d-12a8-4023-9c13-9da55a548d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:33.709885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9de0580d-12a8-4023-9c13-9da55a548d31 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:33.711597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9de0580d-12a8-4023-9c13-9da55a548d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:33.803406 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1204/2372] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:33.811388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e3465-d1e2-4e3f-aeaf-7497205a8c76 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:33.811675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e3465-d1e2-4e3f-aeaf-7497205a8c76 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:33.820425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e3465-d1e2-4e3f-aeaf-7497205a8c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:33.841985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b14e3465-d1e2-4e3f-aeaf-7497205a8c76 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:33.843347 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b14e3465-d1e2-4e3f-aeaf-7497205a8c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:33.873274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1169/2373] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:33.887756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0580d-12a8-4023-9c13-9da55a548d31 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:33.902053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0580d-12a8-4023-9c13-9da55a548d31 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:33.912607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e386-cdb6-431b-a833-cf4ab6559363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:33.913375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e386-cdb6-431b-a833-cf4ab6559363 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:33.913721 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e386-cdb6-431b-a833-cf4ab6559363 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:33.914063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e386-cdb6-431b-a833-cf4ab6559363 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:33.914793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23e386-cdb6-431b-a833-cf4ab6559363 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:52:33.915160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23e386-cdb6-431b-a833-cf4ab6559363 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:33.916866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23e386-cdb6-431b-a833-cf4ab6559363 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:33.931912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1205/2374] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:52:33.934726 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e3465-d1e2-4e3f-aeaf-7497205a8c76 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:33.944773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e3465-d1e2-4e3f-aeaf-7497205a8c76 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:33.949748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aab4b-360f-42b1-929c-77994e41a2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:33.950355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aab4b-360f-42b1-929c-77994e41a2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:33.950640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aab4b-360f-42b1-929c-77994e41a2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:33.950918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aab4b-360f-42b1-929c-77994e41a2e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:33.951566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aab4b-360f-42b1-929c-77994e41a2e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:33.953509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aab4b-360f-42b1-929c-77994e41a2e7 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:34.098498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1170/2375] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:52:33 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:34.102549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e386-cdb6-431b-a833-cf4ab6559363 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:34.112801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e386-cdb6-431b-a833-cf4ab6559363 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:34.170829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cd8e9-3c0c-4b11-a7f0-582ecea61e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:34.177547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cd8e9-3c0c-4b11-a7f0-582ecea61e4c admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:34.177547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cd8e9-3c0c-4b11-a7f0-582ecea61e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:34.177547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cd8e9-3c0c-4b11-a7f0-582ecea61e4c admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:34.177547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cd8e9-3c0c-4b11-a7f0-582ecea61e4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:34.177547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cd8e9-3c0c-4b11-a7f0-582ecea61e4c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:34.300821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1206/2376] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:52:34 2022] POST /identity/v3/users => generated 291 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:34.304876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aab4b-360f-42b1-929c-77994e41a2e7 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:34.315507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aab4b-360f-42b1-929c-77994e41a2e7 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:34.320187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e871a0-834d-4e1c-b274-97e6482fd017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:34.320797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e871a0-834d-4e1c-b274-97e6482fd017 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:34.321083 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e871a0-834d-4e1c-b274-97e6482fd017 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:34.321374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e871a0-834d-4e1c-b274-97e6482fd017 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:34.322041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e871a0-834d-4e1c-b274-97e6482fd017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:34.323668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e871a0-834d-4e1c-b274-97e6482fd017 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:34.334541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1171/2377] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:52:34.338746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cd8e9-3c0c-4b11-a7f0-582ecea61e4c admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:34.349584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cd8e9-3c0c-4b11-a7f0-582ecea61e4c admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:34.354242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3e227-79a2-42d1-8875-3c2b3ce95eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:34.354836 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3e227-79a2-42d1-8875-3c2b3ce95eff admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:34.355115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3e227-79a2-42d1-8875-3c2b3ce95eff admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:34.355389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3e227-79a2-42d1-8875-3c2b3ce95eff admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:34.356068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3e227-79a2-42d1-8875-3c2b3ce95eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:34.357691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3e227-79a2-42d1-8875-3c2b3ce95eff admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:34.368633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1207/2378] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:34.372940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e871a0-834d-4e1c-b274-97e6482fd017 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:34.383275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e871a0-834d-4e1c-b274-97e6482fd017 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:34.387839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0e68a-750e-4907-b62d-c24b4dc85073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:34.388532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0e68a-750e-4907-b62d-c24b4dc85073 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:34.388820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0e68a-750e-4907-b62d-c24b4dc85073 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:34.389101 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0e68a-750e-4907-b62d-c24b4dc85073 admin admin] PATH_INFO: `/v3/projects/7e4bab21dc3141bd89ee2ea54fe0c14c/users/28436612400e421ca52f902542186a68/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:34.404852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0e68a-750e-4907-b62d-c24b4dc85073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bab21dc3141bd89ee2ea54fe0c14c, user_id=28436612400e421ca52f902542186a68, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:34.408812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0e68a-750e-4907-b62d-c24b4dc85073 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:34.531810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1172/2379] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:34 2022] PUT /identity/v3/projects/7e4bab21dc3141bd89ee2ea54fe0c14c/users/28436612400e421ca52f902542186a68/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:34.533573 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e413a-917d-4f12-9a9a-70b0b04796f1 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:34.533979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e413a-917d-4f12-9a9a-70b0b04796f1 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:34.534259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e413a-917d-4f12-9a9a-70b0b04796f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:34.553461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-dc2e413a-917d-4f12-9a9a-70b0b04796f1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:34.554711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-dc2e413a-917d-4f12-9a9a-70b0b04796f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:34.614598 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1208/2380] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:52:34.625872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0e68a-750e-4907-b62d-c24b4dc85073 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:34.628261 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:34.630080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-1fc0e68a-750e-4907-b62d-c24b4dc85073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:34.693735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0e68a-750e-4907-b62d-c24b4dc85073 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:34.697607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5a6fd-6b80-41ca-b2a6-4982303530b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:34.698447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5a6fd-6b80-41ca-b2a6-4982303530b4 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:34.698909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5a6fd-6b80-41ca-b2a6-4982303530b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:34.699189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5a6fd-6b80-41ca-b2a6-4982303530b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:34.717118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5a6fd-6b80-41ca-b2a6-4982303530b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:34.727429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5a6fd-6b80-41ca-b2a6-4982303530b4 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:34.730635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1173/2381] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:35.324521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e413a-917d-4f12-9a9a-70b0b04796f1 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:35.326629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-dc2e413a-917d-4f12-9a9a-70b0b04796f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:35.342236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e413a-917d-4f12-9a9a-70b0b04796f1 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:35.349853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:35.351553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:35.352091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:35.352593 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] PATH_INFO: `/v3/users/78b439283f844e8296f0bd956c93fa81` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:35.353947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b439283f844e8296f0bd956c93fa81)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:35.356785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:35.540386 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:35.662218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b439283f844e8296f0bd956c93fa81'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:35.675483 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b439283f844e8296f0bd956c93fa81'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:35.676596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b439283f844e8296f0bd956c93fa81'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:35.677347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b439283f844e8296f0bd956c93fa81'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:35.783581 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b439283f844e8296f0bd956c93fa81'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:35.796453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1209/2382] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:35 2022] DELETE /identity/v3/users/78b439283f844e8296f0bd956c93fa81 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:35.801007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a34e99-9adf-4af8-a9ef-055afde440c9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:35.801528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a34e99-9adf-4af8-a9ef-055afde440c9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:35.801633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a34e99-9adf-4af8-a9ef-055afde440c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:35.821876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c8a34e99-9adf-4af8-a9ef-055afde440c9 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:35.823155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c8a34e99-9adf-4af8-a9ef-055afde440c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:35.889162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1174/2383] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:35.902565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:35.904503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:35.985505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64130f4e-a812-440b-9e10-0b41f4ec74eb admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:35.989906 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038af8e9-c173-43e8-9f8a-cc70ecf0d2ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:35.990782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af8e9-c173-43e8-9f8a-cc70ecf0d2ea service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:35.991300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af8e9-c173-43e8-9f8a-cc70ecf0d2ea service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:35.991796 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af8e9-c173-43e8-9f8a-cc70ecf0d2ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:36.002899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038af8e9-c173-43e8-9f8a-cc70ecf0d2ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:36.004458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038af8e9-c173-43e8-9f8a-cc70ecf0d2ea service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:36.017845 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1210/2384] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:37.073901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a34e99-9adf-4af8-a9ef-055afde440c9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:37.075400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c8a34e99-9adf-4af8-a9ef-055afde440c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:37.103896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a34e99-9adf-4af8-a9ef-055afde440c9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:37.105857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8419e8-71a6-46f3-bf8b-1ca2036e257c None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:37.106688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8419e8-71a6-46f3-bf8b-1ca2036e257c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:37.107118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8419e8-71a6-46f3-bf8b-1ca2036e257c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:37.130635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:37.138005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:37.138005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:37.138005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] PATH_INFO: `/v3/projects/dd54c4fe43304660b0be26c030016087` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:37.138005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd54c4fe43304660b0be26c030016087)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:37.139583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:37.163942 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-6d8419e8-71a6-46f3-bf8b-1ca2036e257c None None] MFA Rules not processed for user `28436612400e421ca52f902542186a68`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:37.174087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6d8419e8-71a6-46f3-bf8b-1ca2036e257c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:37.266107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1211/2385] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:37.283441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8419e8-71a6-46f3-bf8b-1ca2036e257c None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:37.286303 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6d8419e8-71a6-46f3-bf8b-1ca2036e257c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:37.352418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd54c4fe43304660b0be26c030016087', 'request_id': 'req-94b3c582-a362-404c-93c2-8c92b6e11668'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:37.379129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd54c4fe43304660b0be26c030016087', 'request_id': 'req-94b3c582-a362-404c-93c2-8c92b6e11668'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:37.392010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd54c4fe43304660b0be26c030016087', 'request_id': 'req-94b3c582-a362-404c-93c2-8c92b6e11668'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:37.393155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8419e8-71a6-46f3-bf8b-1ca2036e257c None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:37.485862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] The token cache is being invalidate because project dd54c4fe43304660b0be26c030016087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:37.486523 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:37.489634 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:37.490642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1175/2386] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:37 2022] DELETE /identity/v3/projects/dd54c4fe43304660b0be26c030016087 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:37.494804 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:37.500959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:37.527033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34cddb-0376-4947-8d16-ad86d77bd472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:37.527831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34cddb-0376-4947-8d16-ad86d77bd472 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:37.528265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34cddb-0376-4947-8d16-ad86d77bd472 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:37.528642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34cddb-0376-4947-8d16-ad86d77bd472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:37.535620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-df34cddb-0376-4947-8d16-ad86d77bd472 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:37.644249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3c582-a362-404c-93c2-8c92b6e11668 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:37.652090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a03fa4-0c68-4fcf-8549-20589ee0f9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:37.653330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a03fa4-0c68-4fcf-8549-20589ee0f9bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:37.653810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a03fa4-0c68-4fcf-8549-20589ee0f9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:37.654528 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a03fa4-0c68-4fcf-8549-20589ee0f9bb admin admin] PATH_INFO: `/v3/users/1b3ee4c54bcb4bdfa3ca003de4a5b10a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:37.656189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a03fa4-0c68-4fcf-8549-20589ee0f9bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3ee4c54bcb4bdfa3ca003de4a5b10a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:37.659113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a03fa4-0c68-4fcf-8549-20589ee0f9bb admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:37.677296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34cddb-0376-4947-8d16-ad86d77bd472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:37.679899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34cddb-0376-4947-8d16-ad86d77bd472 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:37.694714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1212/2387] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:37.947004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a7a03fa4-0c68-4fcf-8549-20589ee0f9bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3ee4c54bcb4bdfa3ca003de4a5b10a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:37.947723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a7a03fa4-0c68-4fcf-8549-20589ee0f9bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3ee4c54bcb4bdfa3ca003de4a5b10a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:37.961134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a7a03fa4-0c68-4fcf-8549-20589ee0f9bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3ee4c54bcb4bdfa3ca003de4a5b10a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:38.038086 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a7a03fa4-0c68-4fcf-8549-20589ee0f9bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3ee4c54bcb4bdfa3ca003de4a5b10a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:38.048278 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a7a03fa4-0c68-4fcf-8549-20589ee0f9bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3ee4c54bcb4bdfa3ca003de4a5b10a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:38.054930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1176/2388] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:37 2022] DELETE /identity/v3/users/1b3ee4c54bcb4bdfa3ca003de4a5b10a => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:38.158484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:38.518115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34cddb-0376-4947-8d16-ad86d77bd472 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:38.549078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34cddb-0376-4947-8d16-ad86d77bd472 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:38.615725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:38.616395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:38.616680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:38.616959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] PATH_INFO: `/v3/projects/67c84744199948c98446becfb99dd816` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:38.618092 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67c84744199948c98446becfb99dd816)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:38.620456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:38.843163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67c84744199948c98446becfb99dd816', 'request_id': 'req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:38.935093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:39.167195 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67c84744199948c98446becfb99dd816', 'request_id': 'req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:39.173494 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67c84744199948c98446becfb99dd816', 'request_id': 'req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:39.183584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] The token cache is being invalidate because project 67c84744199948c98446becfb99dd816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:39.183901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:39.184534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-613768ef-6f84-4dc7-b2a5-3b56e84c5ca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:39.186090 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1213/2389] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:38 2022] DELETE /identity/v3/projects/67c84744199948c98446becfb99dd816 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:39.193835 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13a775-c4a6-4fb2-bce0-1fc274c1d028 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:39.194301 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13a775-c4a6-4fb2-bce0-1fc274c1d028 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:39.194425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13a775-c4a6-4fb2-bce0-1fc274c1d028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:39.214231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ff13a775-c4a6-4fb2-bce0-1fc274c1d028 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:39.215453 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ff13a775-c4a6-4fb2-bce0-1fc274c1d028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:39.278871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1177/2390] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:39.287314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6b432-d3b3-48f9-a70b-04147ee801ba None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:39.287901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6b432-d3b3-48f9-a70b-04147ee801ba None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:39.287901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6b432-d3b3-48f9-a70b-04147ee801ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:39.307969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-74c6b432-d3b3-48f9-a70b-04147ee801ba None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:39.309185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-74c6b432-d3b3-48f9-a70b-04147ee801ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:39.318208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1214/2391] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:39.322559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13a775-c4a6-4fb2-bce0-1fc274c1d028 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:39.332262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13a775-c4a6-4fb2-bce0-1fc274c1d028 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:39.336899 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc483de2-f193-4f3a-977e-c4d2d44a899c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:39.337502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc483de2-f193-4f3a-977e-c4d2d44a899c admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:39.337788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc483de2-f193-4f3a-977e-c4d2d44a899c admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:39.338067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc483de2-f193-4f3a-977e-c4d2d44a899c admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:39.338716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc483de2-f193-4f3a-977e-c4d2d44a899c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:52:39.339022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc483de2-f193-4f3a-977e-c4d2d44a899c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:39.340663 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc483de2-f193-4f3a-977e-c4d2d44a899c admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:39.351969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1178/2392] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:52:39.356175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6b432-d3b3-48f9-a70b-04147ee801ba None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:39.365929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6b432-d3b3-48f9-a70b-04147ee801ba None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:39.370580 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6026a99-c8d6-4053-8f41-468da55e5548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:39.371174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6026a99-c8d6-4053-8f41-468da55e5548 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:39.371461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6026a99-c8d6-4053-8f41-468da55e5548 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:39.371742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6026a99-c8d6-4053-8f41-468da55e5548 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:39.372414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6026a99-c8d6-4053-8f41-468da55e5548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:39.374334 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6026a99-c8d6-4053-8f41-468da55e5548 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:39.577685 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1215/2393] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:52:39 2022] POST /identity/v3/projects => generated 375 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:39.582764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc483de2-f193-4f3a-977e-c4d2d44a899c admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:39.593194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc483de2-f193-4f3a-977e-c4d2d44a899c admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:39.668197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79151fc-5e61-4cba-8c0a-4beb18f558db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:39.668796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79151fc-5e61-4cba-8c0a-4beb18f558db admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:39.669109 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79151fc-5e61-4cba-8c0a-4beb18f558db admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:39.669400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79151fc-5e61-4cba-8c0a-4beb18f558db admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:39.670115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79151fc-5e61-4cba-8c0a-4beb18f558db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:39.672901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79151fc-5e61-4cba-8c0a-4beb18f558db admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:39.792794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1179/2394] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:52:39 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:39.797563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6026a99-c8d6-4053-8f41-468da55e5548 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:39.808083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6026a99-c8d6-4053-8f41-468da55e5548 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:39.812504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e629d-466b-4646-b9c1-6c43ec419216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:39.813102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e629d-466b-4646-b9c1-6c43ec419216 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:39.813427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e629d-466b-4646-b9c1-6c43ec419216 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:39.813710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e629d-466b-4646-b9c1-6c43ec419216 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:39.814616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e629d-466b-4646-b9c1-6c43ec419216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:39.816257 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e629d-466b-4646-b9c1-6c43ec419216 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:39.826132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1216/2395] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:39.830138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79151fc-5e61-4cba-8c0a-4beb18f558db admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:39.840168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79151fc-5e61-4cba-8c0a-4beb18f558db admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:39.845456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe965d-92ff-4390-abda-d8e8e832e359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:39.846832 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe965d-92ff-4390-abda-d8e8e832e359 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:39.846832 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe965d-92ff-4390-abda-d8e8e832e359 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:39.846832 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe965d-92ff-4390-abda-d8e8e832e359 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:39.847389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe965d-92ff-4390-abda-d8e8e832e359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:39.849085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe965d-92ff-4390-abda-d8e8e832e359 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:39.860938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1180/2396] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:39.865196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e629d-466b-4646-b9c1-6c43ec419216 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:39.881085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e629d-466b-4646-b9c1-6c43ec419216 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:39.881085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c59ad-c82a-40eb-998d-bea86395e30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:39.881085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c59ad-c82a-40eb-998d-bea86395e30a admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:39.881085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c59ad-c82a-40eb-998d-bea86395e30a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:39.881514 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c59ad-c82a-40eb-998d-bea86395e30a admin admin] PATH_INFO: `/v3/projects/09d9df73d19f4955b822a056d9545a92/users/1c179cd4dc6943419d793a1eb05dc0b5/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:39.898997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c59ad-c82a-40eb-998d-bea86395e30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d9df73d19f4955b822a056d9545a92, user_id=1c179cd4dc6943419d793a1eb05dc0b5, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:39.904121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c59ad-c82a-40eb-998d-bea86395e30a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:39.992174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1217/2397] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:39 2022] PUT /identity/v3/projects/09d9df73d19f4955b822a056d9545a92/users/1c179cd4dc6943419d793a1eb05dc0b5/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:40.004681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5127cb-e494-434c-a2f9-440aab0c2e13 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:40.004972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5127cb-e494-434c-a2f9-440aab0c2e13 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:40.005257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5127cb-e494-434c-a2f9-440aab0c2e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:40.032654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7a5127cb-e494-434c-a2f9-440aab0c2e13 None None] MFA Rules not processed for user `1c179cd4dc6943419d793a1eb05dc0b5`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:40.033886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7a5127cb-e494-434c-a2f9-440aab0c2e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:40.104604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1181/2398] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:40.115931 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c59ad-c82a-40eb-998d-bea86395e30a admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:40.117756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ea9c59ad-c82a-40eb-998d-bea86395e30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:40.121684 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:40.178918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c59ad-c82a-40eb-998d-bea86395e30a admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:40.183228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c0588-c5bf-4b83-a291-368187bb3cac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:40.183359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0588-c5bf-4b83-a291-368187bb3cac service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:40.183545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0588-c5bf-4b83-a291-368187bb3cac service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:40.183818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0588-c5bf-4b83-a291-368187bb3cac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:40.193448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c0588-c5bf-4b83-a291-368187bb3cac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:40.194889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c0588-c5bf-4b83-a291-368187bb3cac service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:40.204520 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1218/2399] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:40.615561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:40.679901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e19f-b5bd-4316-b2df-4aca862bfc79 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:40.680451 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e19f-b5bd-4316-b2df-4aca862bfc79 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:40.680451 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e19f-b5bd-4316-b2df-4aca862bfc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:40.705828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c894e19f-b5bd-4316-b2df-4aca862bfc79 None None] MFA Rules not processed for user `28436612400e421ca52f902542186a68`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:40.705828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c894e19f-b5bd-4316-b2df-4aca862bfc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:40.756591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1182/2400] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:42.114241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ff078-0653-4376-a09e-b9efc58486c4 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:42.115470 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ff078-0653-4376-a09e-b9efc58486c4 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:42.115470 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ff078-0653-4376-a09e-b9efc58486c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:42.134116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-726ff078-0653-4376-a09e-b9efc58486c4 None None] MFA Rules not processed for user `1c179cd4dc6943419d793a1eb05dc0b5`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:42.135248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-726ff078-0653-4376-a09e-b9efc58486c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:42.142801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1219/2401] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:42.289843 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:42.575010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:43.232414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:43.492731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:43.886845 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:44.153264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:44.338946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:44.706755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:44.753833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:45.341083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:45.372781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:45.604842 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:45.629944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:45.884208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:45.914093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:46.215556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:46.284138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:46.467639 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:47.084206 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:47.201782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:47.461164 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:47.718010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:47.875859 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:48.106942 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:48.252747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:48.281843 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:48.758554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:48.867631 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:49.266794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:49.301486 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:49.452968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:49.481561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:49.678475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:49.724167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:49.866740 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:49.895290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:50.015602 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:50.044251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:50.132658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:50.201361 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:50.266586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:50.534372 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:50.564080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:50.670144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:50.747098 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:50.838137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:50.981150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:51.009542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:51.135649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:51.178242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:51.281028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:51.316932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:51.501200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:51.538694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:51.624521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:51.804221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:51.926986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:51.961668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:52.135859 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:52.247173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:59.429611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667ad75-a1a0-484b-8e5b-5fc37d8cf649 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:59.429611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667ad75-a1a0-484b-8e5b-5fc37d8cf649 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:59.430783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667ad75-a1a0-484b-8e5b-5fc37d8cf649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:59.450849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-a667ad75-a1a0-484b-8e5b-5fc37d8cf649 None None] MFA Rules not processed for user `436adf49f3e24e03971017ea6dd33ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:59.452287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a667ad75-a1a0-484b-8e5b-5fc37d8cf649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:59.529168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1183/2402] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:52:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:02.567394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:08.579076 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ff078-0653-4376-a09e-b9efc58486c4 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:08.580740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-726ff078-0653-4376-a09e-b9efc58486c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:08.651262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ff078-0653-4376-a09e-b9efc58486c4 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:08.656033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:08.656617 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:08.656980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:08.657190 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] PATH_INFO: `/v3/users/0b106478ec3b4ea18e01b39621140878` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:08.658331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b106478ec3b4ea18e01b39621140878)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:08.660294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:08.896245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b106478ec3b4ea18e01b39621140878'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:08.908681 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b106478ec3b4ea18e01b39621140878'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:08.909289 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b106478ec3b4ea18e01b39621140878'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:08.909889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b106478ec3b4ea18e01b39621140878'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:09.007342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b106478ec3b4ea18e01b39621140878'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:09.025174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1220/2403] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:53:08 2022] DELETE /identity/v3/users/0b106478ec3b4ea18e01b39621140878 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:09.026564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83e6ce-e633-4ac2-ad0a-a0c8bf1e1657 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:09.027053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83e6ce-e633-4ac2-ad0a-a0c8bf1e1657 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:09.027166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83e6ce-e633-4ac2-ad0a-a0c8bf1e1657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:09.047268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-dd83e6ce-e633-4ac2-ad0a-a0c8bf1e1657 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:09.049117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-dd83e6ce-e633-4ac2-ad0a-a0c8bf1e1657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:09.058660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:09.112825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1184/2404] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:09.122242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:09.125470 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:09.194955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b03718-a0ec-44c7-9420-7e6918635c5f admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:09.199156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03e172-af75-4c07-8f7c-7151343dfd57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:09.199898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03e172-af75-4c07-8f7c-7151343dfd57 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:09.200316 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03e172-af75-4c07-8f7c-7151343dfd57 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:09.200699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03e172-af75-4c07-8f7c-7151343dfd57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:09.212341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03e172-af75-4c07-8f7c-7151343dfd57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:09.213956 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03e172-af75-4c07-8f7c-7151343dfd57 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:09.225868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1221/2405] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:53:09.696601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83e6ce-e633-4ac2-ad0a-a0c8bf1e1657 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:09.698319 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-dd83e6ce-e633-4ac2-ad0a-a0c8bf1e1657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:09.711791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83e6ce-e633-4ac2-ad0a-a0c8bf1e1657 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:09.717218 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:09.718167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:09.718637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:09.719174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] PATH_INFO: `/v3/projects/49903397c23442758a8c7a6505332dd3` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:09.720603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49903397c23442758a8c7a6505332dd3)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:09.722610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:09.856022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49903397c23442758a8c7a6505332dd3', 'request_id': 'req-d509bf04-41ab-4bd8-b42f-3f167f2a7261'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:09.861583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49903397c23442758a8c7a6505332dd3', 'request_id': 'req-d509bf04-41ab-4bd8-b42f-3f167f2a7261'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:09.867662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49903397c23442758a8c7a6505332dd3', 'request_id': 'req-d509bf04-41ab-4bd8-b42f-3f167f2a7261'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:09.957885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] The token cache is being invalidate because project 49903397c23442758a8c7a6505332dd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:53:09.958170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:09.958794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d509bf04-41ab-4bd8-b42f-3f167f2a7261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:09.960317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1185/2406] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:53:09 2022] DELETE /identity/v3/projects/49903397c23442758a8c7a6505332dd3 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:09.965164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171806f-3731-4120-b47d-ef9ffd620fb3 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:09.965655 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171806f-3731-4120-b47d-ef9ffd620fb3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:09.965764 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171806f-3731-4120-b47d-ef9ffd620fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:09.985220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e171806f-3731-4120-b47d-ef9ffd620fb3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:09.986394 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e171806f-3731-4120-b47d-ef9ffd620fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:10.003392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:10.044167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1222/2407] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:10.056388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385c678-439c-46ad-b049-4ca44650c241 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:10.056388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385c678-439c-46ad-b049-4ca44650c241 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:10.056388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385c678-439c-46ad-b049-4ca44650c241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:10.072935 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-2385c678-439c-46ad-b049-4ca44650c241 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:10.074130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-2385c678-439c-46ad-b049-4ca44650c241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:10.081918 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1186/2408] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:10.086168 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171806f-3731-4120-b47d-ef9ffd620fb3 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:10.096008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171806f-3731-4120-b47d-ef9ffd620fb3 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:10.100937 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388a2cd-f3a0-4491-b53a-8c9dc3c7cc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:10.101728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388a2cd-f3a0-4491-b53a-8c9dc3c7cc8c admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:10.102411 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388a2cd-f3a0-4491-b53a-8c9dc3c7cc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:10.102696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388a2cd-f3a0-4491-b53a-8c9dc3c7cc8c admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:10.103444 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388a2cd-f3a0-4491-b53a-8c9dc3c7cc8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:53:10.103746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388a2cd-f3a0-4491-b53a-8c9dc3c7cc8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:10.105388 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388a2cd-f3a0-4491-b53a-8c9dc3c7cc8c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:10.118211 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1223/2409] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:53:10.122570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385c678-439c-46ad-b049-4ca44650c241 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:10.132455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385c678-439c-46ad-b049-4ca44650c241 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:10.137151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362603e-6a32-4716-aa68-dbc59fb59a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:10.137754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362603e-6a32-4716-aa68-dbc59fb59a3d admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:10.138035 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362603e-6a32-4716-aa68-dbc59fb59a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:10.138312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362603e-6a32-4716-aa68-dbc59fb59a3d admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:10.138941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362603e-6a32-4716-aa68-dbc59fb59a3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:10.140871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362603e-6a32-4716-aa68-dbc59fb59a3d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:10.214680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1187/2410] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:53:10 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:53:10.218852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388a2cd-f3a0-4491-b53a-8c9dc3c7cc8c admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:10.229147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388a2cd-f3a0-4491-b53a-8c9dc3c7cc8c admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:10.285744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613ecb1-41a0-4b2a-9dfd-dc24a4a214a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:10.286266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ecb1-41a0-4b2a-9dfd-dc24a4a214a8 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:10.286550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ecb1-41a0-4b2a-9dfd-dc24a4a214a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:10.286825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ecb1-41a0-4b2a-9dfd-dc24a4a214a8 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:10.287455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613ecb1-41a0-4b2a-9dfd-dc24a4a214a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:10.289410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613ecb1-41a0-4b2a-9dfd-dc24a4a214a8 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:10.402779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1224/2411] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:53:10 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:53:10.407091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362603e-6a32-4716-aa68-dbc59fb59a3d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:10.419068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362603e-6a32-4716-aa68-dbc59fb59a3d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:10.444704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bd93b-f6de-492c-b5e2-c900030a9c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:10.445335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bd93b-f6de-492c-b5e2-c900030a9c27 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:10.445620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bd93b-f6de-492c-b5e2-c900030a9c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:10.445892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bd93b-f6de-492c-b5e2-c900030a9c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:10.446545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7bd93b-f6de-492c-b5e2-c900030a9c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:10.448170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7bd93b-f6de-492c-b5e2-c900030a9c27 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:10.461632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1188/2412] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:53:10.465668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613ecb1-41a0-4b2a-9dfd-dc24a4a214a8 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:10.476983 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613ecb1-41a0-4b2a-9dfd-dc24a4a214a8 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:10.481743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd7682-eb4c-49ab-ada3-4010c76ce994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:10.482364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd7682-eb4c-49ab-ada3-4010c76ce994 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:10.482653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd7682-eb4c-49ab-ada3-4010c76ce994 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:10.482934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd7682-eb4c-49ab-ada3-4010c76ce994 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:10.483588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd7682-eb4c-49ab-ada3-4010c76ce994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:10.485250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd7682-eb4c-49ab-ada3-4010c76ce994 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:10.500254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1225/2413] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:53:10.501516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bd93b-f6de-492c-b5e2-c900030a9c27 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:10.511696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bd93b-f6de-492c-b5e2-c900030a9c27 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:10.516986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27bbcf-26a0-48f1-a28b-93e1c2a8cb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:10.517263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27bbcf-26a0-48f1-a28b-93e1c2a8cb6b admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:10.517464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27bbcf-26a0-48f1-a28b-93e1c2a8cb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:10.517743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27bbcf-26a0-48f1-a28b-93e1c2a8cb6b admin admin] PATH_INFO: `/v3/projects/1455f1c4b4704fc28f95725d3d36eb5c/users/151a0ca7370c435b999f78c0dc69b9bc/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:10.532503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27bbcf-26a0-48f1-a28b-93e1c2a8cb6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1455f1c4b4704fc28f95725d3d36eb5c, user_id=151a0ca7370c435b999f78c0dc69b9bc, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:10.535704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27bbcf-26a0-48f1-a28b-93e1c2a8cb6b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:10.648226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1189/2414] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:53:10 2022] PUT /identity/v3/projects/1455f1c4b4704fc28f95725d3d36eb5c/users/151a0ca7370c435b999f78c0dc69b9bc/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:10.660293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695e363-af44-4e01-b7f8-6dccc2e32769 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:10.660587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695e363-af44-4e01-b7f8-6dccc2e32769 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:10.660896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695e363-af44-4e01-b7f8-6dccc2e32769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:10.693900 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f695e363-af44-4e01-b7f8-6dccc2e32769 None None] MFA Rules not processed for user `151a0ca7370c435b999f78c0dc69b9bc`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:10.695201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f695e363-af44-4e01-b7f8-6dccc2e32769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:10.762127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1226/2415] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:10.773769 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27bbcf-26a0-48f1-a28b-93e1c2a8cb6b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:10.775774 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ae27bbcf-26a0-48f1-a28b-93e1c2a8cb6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:10.817558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:10.834466 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27bbcf-26a0-48f1-a28b-93e1c2a8cb6b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:10.842279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564c722-2dd5-495e-a669-f34a7929cbe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:10.842910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564c722-2dd5-495e-a669-f34a7929cbe3 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:10.843190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564c722-2dd5-495e-a669-f34a7929cbe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:10.843464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564c722-2dd5-495e-a669-f34a7929cbe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:10.854217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564c722-2dd5-495e-a669-f34a7929cbe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:10.855730 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564c722-2dd5-495e-a669-f34a7929cbe3 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:10.864844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1190/2416] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:53:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:53:10.890536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695e363-af44-4e01-b7f8-6dccc2e32769 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:10.911173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:10.960207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695e363-af44-4e01-b7f8-6dccc2e32769 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:10.965847 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:10.969458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:10.969458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:10.969458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] PATH_INFO: `/v3/users/151a0ca7370c435b999f78c0dc69b9bc` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:10.969458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=151a0ca7370c435b999f78c0dc69b9bc)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:10.970286 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:11.183517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '151a0ca7370c435b999f78c0dc69b9bc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:11.195962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '151a0ca7370c435b999f78c0dc69b9bc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:11.196633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '151a0ca7370c435b999f78c0dc69b9bc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:11.197245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '151a0ca7370c435b999f78c0dc69b9bc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:11.275923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '151a0ca7370c435b999f78c0dc69b9bc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:11.290646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1227/2417] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:53:10 2022] DELETE /identity/v3/users/151a0ca7370c435b999f78c0dc69b9bc => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:11.295330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5fe8f-1705-423c-a960-07cf4b226bb1 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:11.295786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5fe8f-1705-423c-a960-07cf4b226bb1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:11.295948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5fe8f-1705-423c-a960-07cf4b226bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:11.315665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-28f5fe8f-1705-423c-a960-07cf4b226bb1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:11.317367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-28f5fe8f-1705-423c-a960-07cf4b226bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:11.379576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1191/2418] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:11.389414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:11.390848 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:11.449871 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eb866-25a1-4be8-907f-6a2645f952e2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:11.453397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55d375-6ecf-46be-b7bd-1a4ab0e52937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:11.454023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55d375-6ecf-46be-b7bd-1a4ab0e52937 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:11.454291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55d375-6ecf-46be-b7bd-1a4ab0e52937 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:11.454568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55d375-6ecf-46be-b7bd-1a4ab0e52937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:11.464847 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55d375-6ecf-46be-b7bd-1a4ab0e52937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:11.466331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55d375-6ecf-46be-b7bd-1a4ab0e52937 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:11.477416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1228/2419] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:53:11.994575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5fe8f-1705-423c-a960-07cf4b226bb1 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:11.996061 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-28f5fe8f-1705-423c-a960-07cf4b226bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:12.009288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5fe8f-1705-423c-a960-07cf4b226bb1 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:12.014311 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:12.019414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:12.019414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:12.019414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] PATH_INFO: `/v3/projects/1455f1c4b4704fc28f95725d3d36eb5c` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:12.019414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1455f1c4b4704fc28f95725d3d36eb5c)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:12.019414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:12.238119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1455f1c4b4704fc28f95725d3d36eb5c', 'request_id': 'req-0af3073f-5905-4ef5-91dc-725a1b7ea92d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:12.243632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1455f1c4b4704fc28f95725d3d36eb5c', 'request_id': 'req-0af3073f-5905-4ef5-91dc-725a1b7ea92d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:12.249871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1455f1c4b4704fc28f95725d3d36eb5c', 'request_id': 'req-0af3073f-5905-4ef5-91dc-725a1b7ea92d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:12.367538 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] The token cache is being invalidate because project 1455f1c4b4704fc28f95725d3d36eb5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:53:12.368253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:12.368466 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0af3073f-5905-4ef5-91dc-725a1b7ea92d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:12.369930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1192/2420] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:53:11 2022] DELETE /identity/v3/projects/1455f1c4b4704fc28f95725d3d36eb5c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:12.374814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4efa0f-0b1b-4466-8e89-ea559310a837 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:12.375269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4efa0f-0b1b-4466-8e89-ea559310a837 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:12.375368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4efa0f-0b1b-4466-8e89-ea559310a837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:12.395310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-bf4efa0f-0b1b-4466-8e89-ea559310a837 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:12.396563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bf4efa0f-0b1b-4466-8e89-ea559310a837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:12.429932 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:12.457252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1229/2421] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:12.465098 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a461-ff75-403e-901b-70f59faea980 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:12.465385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a461-ff75-403e-901b-70f59faea980 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:12.466112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a461-ff75-403e-901b-70f59faea980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:12.486513 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-cc91a461-ff75-403e-901b-70f59faea980 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:12.487723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cc91a461-ff75-403e-901b-70f59faea980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:12.500739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1193/2422] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:12.500903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4efa0f-0b1b-4466-8e89-ea559310a837 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:12.510158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4efa0f-0b1b-4466-8e89-ea559310a837 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:12.514800 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca39275-b9d3-417c-9c99-d6d1b6f7709f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:12.515395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca39275-b9d3-417c-9c99-d6d1b6f7709f admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:12.515677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca39275-b9d3-417c-9c99-d6d1b6f7709f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:12.515986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca39275-b9d3-417c-9c99-d6d1b6f7709f admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:12.516634 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca39275-b9d3-417c-9c99-d6d1b6f7709f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:53:12.516965 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca39275-b9d3-417c-9c99-d6d1b6f7709f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:12.518562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca39275-b9d3-417c-9c99-d6d1b6f7709f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:12.529926 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1230/2423] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:53:12.555185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91a461-ff75-403e-901b-70f59faea980 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:12.565276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91a461-ff75-403e-901b-70f59faea980 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:12.571246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9399b-e12a-416d-a340-19bed895363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:12.571855 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9399b-e12a-416d-a340-19bed895363d admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:12.572174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9399b-e12a-416d-a340-19bed895363d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:12.572480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9399b-e12a-416d-a340-19bed895363d admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:12.573131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9399b-e12a-416d-a340-19bed895363d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:12.575055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9399b-e12a-416d-a340-19bed895363d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:12.693201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1194/2424] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:53:12 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:53:12.694262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca39275-b9d3-417c-9c99-d6d1b6f7709f admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:12.705371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca39275-b9d3-417c-9c99-d6d1b6f7709f admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:12.779614 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b12f40-e03d-45ef-ab1e-2a769612b97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:12.780506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b12f40-e03d-45ef-ab1e-2a769612b97a admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:12.780787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b12f40-e03d-45ef-ab1e-2a769612b97a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:12.781064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b12f40-e03d-45ef-ab1e-2a769612b97a admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:12.781698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b12f40-e03d-45ef-ab1e-2a769612b97a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:12.784046 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b12f40-e03d-45ef-ab1e-2a769612b97a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:12.907420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1231/2425] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:53:12 2022] POST /identity/v3/users => generated 288 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:53:12.916429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9399b-e12a-416d-a340-19bed895363d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:12.927816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9399b-e12a-416d-a340-19bed895363d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:12.938783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc3f44-2e36-4b45-86c0-f1cd71ebf093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:12.939963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc3f44-2e36-4b45-86c0-f1cd71ebf093 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:12.940246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc3f44-2e36-4b45-86c0-f1cd71ebf093 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:12.940770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc3f44-2e36-4b45-86c0-f1cd71ebf093 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:12.941427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc3f44-2e36-4b45-86c0-f1cd71ebf093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:12.943017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc3f44-2e36-4b45-86c0-f1cd71ebf093 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:12.958192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1195/2426] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:53:12.965075 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b12f40-e03d-45ef-ab1e-2a769612b97a admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:12.975338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b12f40-e03d-45ef-ab1e-2a769612b97a admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:12.980100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc8ad6-eb71-4336-b357-e9edd4d48703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:12.980704 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc8ad6-eb71-4336-b357-e9edd4d48703 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:12.980986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc8ad6-eb71-4336-b357-e9edd4d48703 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:12.981260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc8ad6-eb71-4336-b357-e9edd4d48703 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:12.981912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc8ad6-eb71-4336-b357-e9edd4d48703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:12.983498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc8ad6-eb71-4336-b357-e9edd4d48703 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:12.997825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1232/2427] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:53:12.998942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc3f44-2e36-4b45-86c0-f1cd71ebf093 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:13.009943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc3f44-2e36-4b45-86c0-f1cd71ebf093 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:13.014655 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd11bf-f2af-4127-87d4-835db40376f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:13.015310 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd11bf-f2af-4127-87d4-835db40376f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:13.015591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd11bf-f2af-4127-87d4-835db40376f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:13.015886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd11bf-f2af-4127-87d4-835db40376f2 admin admin] PATH_INFO: `/v3/projects/4e7e93800ed64c899441aed574e4a7b6/users/5f97decedf434e74b94572ed03b86186/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:13.037450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd11bf-f2af-4127-87d4-835db40376f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e7e93800ed64c899441aed574e4a7b6, user_id=5f97decedf434e74b94572ed03b86186, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:13.041818 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd11bf-f2af-4127-87d4-835db40376f2 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:13.148204 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:13.155129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1196/2428] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:53:12 2022] PUT /identity/v3/projects/4e7e93800ed64c899441aed574e4a7b6/users/5f97decedf434e74b94572ed03b86186/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:13.165696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45990dd4-9e6c-4fe6-9fb5-1b1d59b5ff55 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:13.165979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45990dd4-9e6c-4fe6-9fb5-1b1d59b5ff55 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:13.166474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45990dd4-9e6c-4fe6-9fb5-1b1d59b5ff55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:13.199048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-45990dd4-9e6c-4fe6-9fb5-1b1d59b5ff55 None None] MFA Rules not processed for user `5f97decedf434e74b94572ed03b86186`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:13.200805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-45990dd4-9e6c-4fe6-9fb5-1b1d59b5ff55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:13.270392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1233/2429] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:13.281221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd11bf-f2af-4127-87d4-835db40376f2 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:13.283924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0ecd11bf-f2af-4127-87d4-835db40376f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:13.343944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd11bf-f2af-4127-87d4-835db40376f2 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:13.348684 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27942258-cf8b-4e2b-94ee-ef4163f1062e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:13.349346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27942258-cf8b-4e2b-94ee-ef4163f1062e service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:13.349632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27942258-cf8b-4e2b-94ee-ef4163f1062e service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:13.349908 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27942258-cf8b-4e2b-94ee-ef4163f1062e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:13.358867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27942258-cf8b-4e2b-94ee-ef4163f1062e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:13.360438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27942258-cf8b-4e2b-94ee-ef4163f1062e service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:13.370384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1197/2430] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:53:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:53:16.923143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45990dd4-9e6c-4fe6-9fb5-1b1d59b5ff55 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:16.925184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-45990dd4-9e6c-4fe6-9fb5-1b1d59b5ff55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:16.966203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:16.995132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45990dd4-9e6c-4fe6-9fb5-1b1d59b5ff55 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:17.000393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f371d-c776-465d-bd71-2988de751af2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:17.001355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f371d-c776-465d-bd71-2988de751af2 service placement] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:17.002270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f371d-c776-465d-bd71-2988de751af2 service placement] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:17.002553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f371d-c776-465d-bd71-2988de751af2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:17.005141 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a74f371d-c776-465d-bd71-2988de751af2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:17.017793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f371d-c776-465d-bd71-2988de751af2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:17.019534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f371d-c776-465d-bd71-2988de751af2 service placement] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:17.030374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1234/2431] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:53:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:53:18.098907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb636a10-25cb-457f-8a35-6e1ec0d98985 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:18.099428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb636a10-25cb-457f-8a35-6e1ec0d98985 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:18.099428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb636a10-25cb-457f-8a35-6e1ec0d98985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:18.119532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-cb636a10-25cb-457f-8a35-6e1ec0d98985 None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:18.120959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cb636a10-25cb-457f-8a35-6e1ec0d98985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:18.182284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1198/2432] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:53:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:18.227382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f371d-c776-465d-bd71-2988de751af2 service placement] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:18.229132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a74f371d-c776-465d-bd71-2988de751af2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:18.299652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f371d-c776-465d-bd71-2988de751af2 service placement] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:18.305380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc235d-e778-4e67-ad82-0551ab2b113b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:18.305995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc235d-e778-4e67-ad82-0551ab2b113b service swift] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:18.306278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc235d-e778-4e67-ad82-0551ab2b113b service swift] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:18.306553 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc235d-e778-4e67-ad82-0551ab2b113b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:18.317173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc235d-e778-4e67-ad82-0551ab2b113b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:18.318683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc235d-e778-4e67-ad82-0551ab2b113b service swift] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:18.329452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1235/2433] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:53:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:53:19.389346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e9fd5-1dd1-4ab2-8c0e-a8e53b1fb3d7 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:19.389883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e9fd5-1dd1-4ab2-8c0e-a8e53b1fb3d7 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:19.389883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e9fd5-1dd1-4ab2-8c0e-a8e53b1fb3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:19.416249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-645e9fd5-1dd1-4ab2-8c0e-a8e53b1fb3d7 None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:19.417784 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-645e9fd5-1dd1-4ab2-8c0e-a8e53b1fb3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:19.425347 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1199/2434] 10.222.0.98 () {58 vars in 967 bytes} [Fri Aug 26 15:53:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:19.438846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc235d-e778-4e67-ad82-0551ab2b113b service swift] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:19.450530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc235d-e778-4e67-ad82-0551ab2b113b service swift] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:19.456403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91c323-316f-468c-b71c-2a001e55d2e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:19.457008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91c323-316f-468c-b71c-2a001e55d2e9 service swift] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:19.457293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91c323-316f-468c-b71c-2a001e55d2e9 service swift] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:19.457575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91c323-316f-468c-b71c-2a001e55d2e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:19.466768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91c323-316f-468c-b71c-2a001e55d2e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:19.469305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91c323-316f-468c-b71c-2a001e55d2e9 service swift] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:19.478950 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1236/2435] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:53:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:53:20.368509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda517fe-ed30-4840-b860-05f36630224b None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:20.368509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda517fe-ed30-4840-b860-05f36630224b None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:20.368509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda517fe-ed30-4840-b860-05f36630224b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:20.389017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-dda517fe-ed30-4840-b860-05f36630224b None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:20.390567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-dda517fe-ed30-4840-b860-05f36630224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:20.414690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1200/2436] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:53:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:20.427744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91c323-316f-468c-b71c-2a001e55d2e9 service swift] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:20.443114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91c323-316f-468c-b71c-2a001e55d2e9 service swift] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:20.453261 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285963a-0cf4-4137-9ea1-8ff207f38869 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:20.453881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285963a-0cf4-4137-9ea1-8ff207f38869 service swift] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:20.454166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285963a-0cf4-4137-9ea1-8ff207f38869 service swift] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:20.454626 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285963a-0cf4-4137-9ea1-8ff207f38869 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:20.464518 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285963a-0cf4-4137-9ea1-8ff207f38869 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:20.466267 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285963a-0cf4-4137-9ea1-8ff207f38869 service swift] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:20.475742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1237/2437] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:53:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:53:21.178066 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb4167-d9ac-41bb-b129-f17a5179728a None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:21.178359 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb4167-d9ac-41bb-b129-f17a5179728a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:21.178641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb4167-d9ac-41bb-b129-f17a5179728a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:21.201322 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-abcb4167-d9ac-41bb-b129-f17a5179728a None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:21.202573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-abcb4167-d9ac-41bb-b129-f17a5179728a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:21.212457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1201/2438] 10.222.0.98 () {58 vars in 967 bytes} [Fri Aug 26 15:53:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:53:21.232322 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285963a-0cf4-4137-9ea1-8ff207f38869 service swift] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:21.243617 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285963a-0cf4-4137-9ea1-8ff207f38869 service swift] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:21.248829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ab6cb-93ab-40f4-9768-acab4c5e8ac0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:21.249473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ab6cb-93ab-40f4-9768-acab4c5e8ac0 service swift] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:21.249759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ab6cb-93ab-40f4-9768-acab4c5e8ac0 service swift] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:21.250037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ab6cb-93ab-40f4-9768-acab4c5e8ac0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:21.259255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ab6cb-93ab-40f4-9768-acab4c5e8ac0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:21.260914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ab6cb-93ab-40f4-9768-acab4c5e8ac0 service swift] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:21.270228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1238/2439] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:53:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:53:21.678784 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb4167-d9ac-41bb-b129-f17a5179728a None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:21.683991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-abcb4167-d9ac-41bb-b129-f17a5179728a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:21.760858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb4167-d9ac-41bb-b129-f17a5179728a None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:21.765587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d82788-2aa4-4824-b97e-55df560f95bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:21.766436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d82788-2aa4-4824-b97e-55df560f95bf service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:21.766842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d82788-2aa4-4824-b97e-55df560f95bf service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:21.767246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d82788-2aa4-4824-b97e-55df560f95bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:21.769491 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-01d82788-2aa4-4824-b97e-55df560f95bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:21.836563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:21.862291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d82788-2aa4-4824-b97e-55df560f95bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:21.863660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d82788-2aa4-4824-b97e-55df560f95bf service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:21.880088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1202/2440] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:53:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:53:24.202420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ab6cb-93ab-40f4-9768-acab4c5e8ac0 service swift] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:24.202420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-223ab6cb-93ab-40f4-9768-acab4c5e8ac0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:24.297346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ab6cb-93ab-40f4-9768-acab4c5e8ac0 service swift] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:24.303319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:24.304157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:24.304447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:24.304727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] PATH_INFO: `/v3/users/1c179cd4dc6943419d793a1eb05dc0b5` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:24.306168 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c179cd4dc6943419d793a1eb05dc0b5)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:24.309185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:24.333113 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:24.580331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c179cd4dc6943419d793a1eb05dc0b5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:24.613979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c179cd4dc6943419d793a1eb05dc0b5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:24.615034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c179cd4dc6943419d793a1eb05dc0b5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:24.615938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c179cd4dc6943419d793a1eb05dc0b5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:24.773741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c179cd4dc6943419d793a1eb05dc0b5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:24.789524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1239/2441] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:53:24 2022] DELETE /identity/v3/users/1c179cd4dc6943419d793a1eb05dc0b5 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:24.792437 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb755d5-6293-47cf-ab15-6d9f805389ab None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:24.792938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb755d5-6293-47cf-ab15-6d9f805389ab None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:24.793039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb755d5-6293-47cf-ab15-6d9f805389ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:24.813476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-7fb755d5-6293-47cf-ab15-6d9f805389ab None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:24.814968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7fb755d5-6293-47cf-ab15-6d9f805389ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:24.882925 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1203/2442] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:53:24.892702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:24.894228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:24.976318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99405f-3b3a-4e41-b541-360b951dd345 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:24.991063 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79f14f-16bf-41d2-99be-cf9ca26734bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:24.992756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79f14f-16bf-41d2-99be-cf9ca26734bd service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:24.992756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79f14f-16bf-41d2-99be-cf9ca26734bd service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:25.000317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79f14f-16bf-41d2-99be-cf9ca26734bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:25.015895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79f14f-16bf-41d2-99be-cf9ca26734bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:25.019592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79f14f-16bf-41d2-99be-cf9ca26734bd service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:25.030458 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1240/2443] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:53:25.269641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb755d5-6293-47cf-ab15-6d9f805389ab None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:25.271265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7fb755d5-6293-47cf-ab15-6d9f805389ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:25.288693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb755d5-6293-47cf-ab15-6d9f805389ab None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:25.294615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:25.295434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:25.295888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:25.305638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] PATH_INFO: `/v3/projects/09d9df73d19f4955b822a056d9545a92` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:25.305638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d9df73d19f4955b822a056d9545a92)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:25.305638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:25.402420 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d9df73d19f4955b822a056d9545a92', 'request_id': 'req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:25.408717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d9df73d19f4955b822a056d9545a92', 'request_id': 'req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:25.415017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d9df73d19f4955b822a056d9545a92', 'request_id': 'req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:25.493102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] The token cache is being invalidate because project 09d9df73d19f4955b822a056d9545a92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:53:25.493537 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:25.494139 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-8f8337d8-91f0-4fb3-9fc4-fc2e45f40f05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:25.495607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1204/2444] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:53:25 2022] DELETE /identity/v3/projects/09d9df73d19f4955b822a056d9545a92 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:26.094829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79f14f-16bf-41d2-99be-cf9ca26734bd service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:26.096452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1b79f14f-16bf-41d2-99be-cf9ca26734bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:26.110346 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79f14f-16bf-41d2-99be-cf9ca26734bd service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:26.169759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:26.170290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:26.170576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:26.170854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] PATH_INFO: `/v3/users/5f97decedf434e74b94572ed03b86186` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:26.172139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f97decedf434e74b94572ed03b86186)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:26.174111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:26.399857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:26.405858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f97decedf434e74b94572ed03b86186'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:26.417811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f97decedf434e74b94572ed03b86186'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:26.418523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f97decedf434e74b94572ed03b86186'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:26.419135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f97decedf434e74b94572ed03b86186'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:26.522127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f97decedf434e74b94572ed03b86186'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:26.535078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1241/2445] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:53:26 2022] DELETE /identity/v3/users/5f97decedf434e74b94572ed03b86186 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:26.539699 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213b662-7bcd-44f6-878e-4876fb11ae74 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:26.540258 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213b662-7bcd-44f6-878e-4876fb11ae74 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:26.540364 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213b662-7bcd-44f6-878e-4876fb11ae74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:26.559856 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-6213b662-7bcd-44f6-878e-4876fb11ae74 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:26.561085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6213b662-7bcd-44f6-878e-4876fb11ae74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:26.616519 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:26.624865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1205/2446] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:26.635372 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:26.636907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:26.701906 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d731ac-8c97-42b7-8546-2aa1bb5b3453 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:26.705693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680d88e-995a-4853-9f75-d89ee155a09a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:26.706302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d88e-995a-4853-9f75-d89ee155a09a service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:26.706589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d88e-995a-4853-9f75-d89ee155a09a service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:26.706863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d88e-995a-4853-9f75-d89ee155a09a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:26.716488 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680d88e-995a-4853-9f75-d89ee155a09a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:26.717918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680d88e-995a-4853-9f75-d89ee155a09a service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:26.727746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1242/2447] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:53:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:53:27.195211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213b662-7bcd-44f6-878e-4876fb11ae74 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:27.196767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6213b662-7bcd-44f6-878e-4876fb11ae74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:27.211237 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213b662-7bcd-44f6-878e-4876fb11ae74 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:27.216269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:27.217292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:27.217760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:27.218040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] PATH_INFO: `/v3/projects/4e7e93800ed64c899441aed574e4a7b6` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:27.219289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e7e93800ed64c899441aed574e4a7b6)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:27.222122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:27.359760 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e7e93800ed64c899441aed574e4a7b6', 'request_id': 'req-5bea6804-90e5-44dc-93b2-6f7a92395012'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:27.365627 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e7e93800ed64c899441aed574e4a7b6', 'request_id': 'req-5bea6804-90e5-44dc-93b2-6f7a92395012'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:27.372313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e7e93800ed64c899441aed574e4a7b6', 'request_id': 'req-5bea6804-90e5-44dc-93b2-6f7a92395012'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:27.476613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] The token cache is being invalidate because project 4e7e93800ed64c899441aed574e4a7b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:53:27.476905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:27.477566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5bea6804-90e5-44dc-93b2-6f7a92395012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:53:27.479053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1206/2448] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:53:27 2022] DELETE /identity/v3/projects/4e7e93800ed64c899441aed574e4a7b6 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:27.484464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b3f6a-e24b-453c-bb66-02ad393a9366 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:27.484891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b3f6a-e24b-453c-bb66-02ad393a9366 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:27.484891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b3f6a-e24b-453c-bb66-02ad393a9366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:27.504508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-4b3b3f6a-e24b-453c-bb66-02ad393a9366 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:27.505725 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4b3b3f6a-e24b-453c-bb66-02ad393a9366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:27.567263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1243/2449] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:27.581645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc3b7b-54e0-4805-9e43-f8dbf03f4965 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:27.582153 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc3b7b-54e0-4805-9e43-f8dbf03f4965 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:27.582276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc3b7b-54e0-4805-9e43-f8dbf03f4965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:27.602577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-a1fc3b7b-54e0-4805-9e43-f8dbf03f4965 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:27.603808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a1fc3b7b-54e0-4805-9e43-f8dbf03f4965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:27.612066 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1207/2450] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:27.616235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b3f6a-e24b-453c-bb66-02ad393a9366 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:27.626068 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b3f6a-e24b-453c-bb66-02ad393a9366 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:27.630755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271dec0-160a-4102-931d-74f2002fcb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:27.631349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271dec0-160a-4102-931d-74f2002fcb74 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:27.631634 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271dec0-160a-4102-931d-74f2002fcb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:27.631931 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271dec0-160a-4102-931d-74f2002fcb74 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:27.632579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271dec0-160a-4102-931d-74f2002fcb74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:53:27.632885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271dec0-160a-4102-931d-74f2002fcb74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:27.634648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271dec0-160a-4102-931d-74f2002fcb74 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:27.648415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1244/2451] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:53:27.652947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc3b7b-54e0-4805-9e43-f8dbf03f4965 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:27.663126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc3b7b-54e0-4805-9e43-f8dbf03f4965 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:27.667853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1909b9c-368b-475f-ab45-b0a8c894be31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:27.668478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1909b9c-368b-475f-ab45-b0a8c894be31 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:27.671976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1909b9c-368b-475f-ab45-b0a8c894be31 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:27.671976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1909b9c-368b-475f-ab45-b0a8c894be31 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:27.671976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1909b9c-368b-475f-ab45-b0a8c894be31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:27.671976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1909b9c-368b-475f-ab45-b0a8c894be31 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:27.803442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1208/2452] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:53:27 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:53:27.808126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271dec0-160a-4102-931d-74f2002fcb74 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:27.819222 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271dec0-160a-4102-931d-74f2002fcb74 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:27.866217 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:27.879568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b72505-5fc5-43f1-803a-5f27bdbbccea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:27.880331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b72505-5fc5-43f1-803a-5f27bdbbccea admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:27.880619 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b72505-5fc5-43f1-803a-5f27bdbbccea admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:27.880899 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b72505-5fc5-43f1-803a-5f27bdbbccea admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:27.881535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b72505-5fc5-43f1-803a-5f27bdbbccea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:27.883507 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b72505-5fc5-43f1-803a-5f27bdbbccea admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:27.974138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1245/2453] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:53:27 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:53:27.975315 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1909b9c-368b-475f-ab45-b0a8c894be31 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:27.985584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1909b9c-368b-475f-ab45-b0a8c894be31 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:27.990266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cc450-8016-4a70-bbd4-f44cb94a1b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:27.990937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cc450-8016-4a70-bbd4-f44cb94a1b89 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:27.991223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cc450-8016-4a70-bbd4-f44cb94a1b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:27.991504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cc450-8016-4a70-bbd4-f44cb94a1b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:27.992229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cc450-8016-4a70-bbd4-f44cb94a1b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:27.993818 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cc450-8016-4a70-bbd4-f44cb94a1b89 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:28.004983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1209/2454] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:53:28.009259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b72505-5fc5-43f1-803a-5f27bdbbccea admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:28.020378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b72505-5fc5-43f1-803a-5f27bdbbccea admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:28.025967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64820be4-9253-4c38-80ec-7d616cf1cab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:28.026566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64820be4-9253-4c38-80ec-7d616cf1cab8 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:28.026885 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64820be4-9253-4c38-80ec-7d616cf1cab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:28.031426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64820be4-9253-4c38-80ec-7d616cf1cab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:28.031426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64820be4-9253-4c38-80ec-7d616cf1cab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:28.031426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64820be4-9253-4c38-80ec-7d616cf1cab8 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:28.041344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1246/2455] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:53:28.045579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cc450-8016-4a70-bbd4-f44cb94a1b89 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:28.057988 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cc450-8016-4a70-bbd4-f44cb94a1b89 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:28.063362 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba68640-57ad-4299-b625-f1f781f58eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:28.064923 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba68640-57ad-4299-b625-f1f781f58eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:28.065264 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba68640-57ad-4299-b625-f1f781f58eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:28.065567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba68640-57ad-4299-b625-f1f781f58eb5 admin admin] PATH_INFO: `/v3/projects/1b64a7d4a15a4d019f7bfc17a59889a5/users/b0eb3bddc1bd4557a280ad21f9092093/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:28.085434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba68640-57ad-4299-b625-f1f781f58eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b64a7d4a15a4d019f7bfc17a59889a5, user_id=b0eb3bddc1bd4557a280ad21f9092093, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:28.088156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba68640-57ad-4299-b625-f1f781f58eb5 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:28.194693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1210/2456] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:53:28 2022] PUT /identity/v3/projects/1b64a7d4a15a4d019f7bfc17a59889a5/users/b0eb3bddc1bd4557a280ad21f9092093/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:28.199953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174110-f8fd-4eb9-9652-87851d1a843a None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:28.200249 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174110-f8fd-4eb9-9652-87851d1a843a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:28.200532 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174110-f8fd-4eb9-9652-87851d1a843a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:28.219608 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ed174110-f8fd-4eb9-9652-87851d1a843a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:28.222465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ed174110-f8fd-4eb9-9652-87851d1a843a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:28.286070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1247/2457] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:28.295974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba68640-57ad-4299-b625-f1f781f58eb5 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:28.297714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-eba68640-57ad-4299-b625-f1f781f58eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:28.358960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba68640-57ad-4299-b625-f1f781f58eb5 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:28.363297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34addcab-1daf-4aca-8781-8cebdc2dec1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:28.363937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34addcab-1daf-4aca-8781-8cebdc2dec1e service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:28.364223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34addcab-1daf-4aca-8781-8cebdc2dec1e service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:28.364494 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34addcab-1daf-4aca-8781-8cebdc2dec1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:28.375475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34addcab-1daf-4aca-8781-8cebdc2dec1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:28.377511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34addcab-1daf-4aca-8781-8cebdc2dec1e service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:28.388593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1211/2458] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:53:32.645315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6680-ee92-492a-972a-ec2d9465f129 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:32.645811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6680-ee92-492a-972a-ec2d9465f129 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:32.646304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6680-ee92-492a-972a-ec2d9465f129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:32.672834 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9ece6680-ee92-492a-972a-ec2d9465f129 None None] MFA Rules not processed for user `9348310fd06b473786d1c6be4f20c481`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:32.674522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9ece6680-ee92-492a-972a-ec2d9465f129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:32.769293 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1248/2459] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:35.680589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777dff7-75ac-4a5a-810e-f89abfd82269 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:35.681466 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777dff7-75ac-4a5a-810e-f89abfd82269 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:35.681907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777dff7-75ac-4a5a-810e-f89abfd82269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:35.714966 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-0777dff7-75ac-4a5a-810e-f89abfd82269 None None] MFA Rules not processed for user `b0eb3bddc1bd4557a280ad21f9092093`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:35.716262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0777dff7-75ac-4a5a-810e-f89abfd82269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:35.729791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:35.807735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1212/2460] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:35.812088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece6680-ee92-492a-972a-ec2d9465f129 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:35.825998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece6680-ee92-492a-972a-ec2d9465f129 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:35.831338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e03412-1d62-43f3-917c-ace5bd9e8204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:35.831997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e03412-1d62-43f3-917c-ace5bd9e8204 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:35.832288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e03412-1d62-43f3-917c-ace5bd9e8204 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:35.832566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e03412-1d62-43f3-917c-ace5bd9e8204 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:35.833216 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e03412-1d62-43f3-917c-ace5bd9e8204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:35.835241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e03412-1d62-43f3-917c-ace5bd9e8204 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:35.974415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1249/2461] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:53:35 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:53:35.982733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777dff7-75ac-4a5a-810e-f89abfd82269 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:35.996089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777dff7-75ac-4a5a-810e-f89abfd82269 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:36.077276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01689b30-bef4-4f5c-bea0-10f8b06e3e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:36.077824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01689b30-bef4-4f5c-bea0-10f8b06e3e60 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:36.078111 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01689b30-bef4-4f5c-bea0-10f8b06e3e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:36.078387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01689b30-bef4-4f5c-bea0-10f8b06e3e60 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:36.079018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01689b30-bef4-4f5c-bea0-10f8b06e3e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:36.081799 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01689b30-bef4-4f5c-bea0-10f8b06e3e60 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:36.200965 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1213/2462] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:53:35 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:53:36.206001 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e03412-1d62-43f3-917c-ace5bd9e8204 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:36.217175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e03412-1d62-43f3-917c-ace5bd9e8204 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:36.221964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662b61c-a792-4247-8bb9-23616b0f64e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:36.222651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662b61c-a792-4247-8bb9-23616b0f64e2 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:36.222849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662b61c-a792-4247-8bb9-23616b0f64e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:36.223132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662b61c-a792-4247-8bb9-23616b0f64e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:36.223786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662b61c-a792-4247-8bb9-23616b0f64e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:36.225639 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662b61c-a792-4247-8bb9-23616b0f64e2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:36.237635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1250/2463] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:53:36.241888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01689b30-bef4-4f5c-bea0-10f8b06e3e60 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:36.255427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01689b30-bef4-4f5c-bea0-10f8b06e3e60 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:36.261142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0819129-f19f-4249-805d-2e37ddb7eeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:36.261776 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0819129-f19f-4249-805d-2e37ddb7eeca admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:36.262092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0819129-f19f-4249-805d-2e37ddb7eeca admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:36.262380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0819129-f19f-4249-805d-2e37ddb7eeca admin admin] PATH_INFO: `/v3/projects/f357910be96c43b1bbdea77c6f9d9152/users/496f6511485d476baddb2ea38531e772/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:36.281328 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0819129-f19f-4249-805d-2e37ddb7eeca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f357910be96c43b1bbdea77c6f9d9152, user_id=496f6511485d476baddb2ea38531e772, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:36.284121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0819129-f19f-4249-805d-2e37ddb7eeca admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:36.385896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1214/2464] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:53:36 2022] PUT /identity/v3/projects/f357910be96c43b1bbdea77c6f9d9152/users/496f6511485d476baddb2ea38531e772/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:36.390310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662b61c-a792-4247-8bb9-23616b0f64e2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:36.403105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662b61c-a792-4247-8bb9-23616b0f64e2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:36.488561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:36.532909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527df445-2268-43f3-9aea-7f99ad29fb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:36.533568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527df445-2268-43f3-9aea-7f99ad29fb74 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:36.539265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527df445-2268-43f3-9aea-7f99ad29fb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:36.539265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527df445-2268-43f3-9aea-7f99ad29fb74 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:36.539265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527df445-2268-43f3-9aea-7f99ad29fb74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:36.539265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527df445-2268-43f3-9aea-7f99ad29fb74 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:36.574471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0819129-f19f-4249-805d-2e37ddb7eeca admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:36.575527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1251/2465] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:53:36.600188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0819129-f19f-4249-805d-2e37ddb7eeca admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:36.607654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf2b42-3dde-49f1-8b4c-bef9791fe84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:36.608795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf2b42-3dde-49f1-8b4c-bef9791fe84b admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:36.609268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf2b42-3dde-49f1-8b4c-bef9791fe84b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:36.609727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf2b42-3dde-49f1-8b4c-bef9791fe84b admin admin] PATH_INFO: `/v3/projects/f357910be96c43b1bbdea77c6f9d9152/users/496f6511485d476baddb2ea38531e772/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:36.612739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf2b42-3dde-49f1-8b4c-bef9791fe84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f357910be96c43b1bbdea77c6f9d9152, user_id=496f6511485d476baddb2ea38531e772, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:36.616132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf2b42-3dde-49f1-8b4c-bef9791fe84b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:36.630384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1215/2466] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:53:36 2022] PUT /identity/v3/projects/f357910be96c43b1bbdea77c6f9d9152/users/496f6511485d476baddb2ea38531e772/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:44.676904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218e1f-9a62-41ce-88d1-c6c83a205f86 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:44.677337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218e1f-9a62-41ce-88d1-c6c83a205f86 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:44.677446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218e1f-9a62-41ce-88d1-c6c83a205f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:44.707297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-cb218e1f-9a62-41ce-88d1-c6c83a205f86 None None] MFA Rules not processed for user `496f6511485d476baddb2ea38531e772`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:44.708779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cb218e1f-9a62-41ce-88d1-c6c83a205f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:44.783613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1252/2467] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:44.795955 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf2b42-3dde-49f1-8b4c-bef9791fe84b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:44.797669 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0adf2b42-3dde-49f1-8b4c-bef9791fe84b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:44.856588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf2b42-3dde-49f1-8b4c-bef9791fe84b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:44.860147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd849de3-c2b7-4af3-abb1-ad6b4aea8571 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cef193a54eff44f2b53716e3ec1a29cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:44.860753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd849de3-c2b7-4af3-abb1-ad6b4aea8571 service glance] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:44.861029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd849de3-c2b7-4af3-abb1-ad6b4aea8571 service glance] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:44.861812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd849de3-c2b7-4af3-abb1-ad6b4aea8571 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:44.879275 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:44.921755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd849de3-c2b7-4af3-abb1-ad6b4aea8571 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:44.923194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd849de3-c2b7-4af3-abb1-ad6b4aea8571 service glance] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:44.933436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1216/2468] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:53:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:53:45.186142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd753af-f0ff-406b-852a-3a38f557bb70 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:45.186429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd753af-f0ff-406b-852a-3a38f557bb70 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:45.186708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd753af-f0ff-406b-852a-3a38f557bb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:45.206555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-8dd753af-f0ff-406b-852a-3a38f557bb70 None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:45.207667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8dd753af-f0ff-406b-852a-3a38f557bb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:45.270323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1253/2469] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:53:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:53:45.282666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd849de3-c2b7-4af3-abb1-ad6b4aea8571 service glance] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:53:45.284859 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-bd849de3-c2b7-4af3-abb1-ad6b4aea8571 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:45.355091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd849de3-c2b7-4af3-abb1-ad6b4aea8571 service glance] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:53:45.361323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b06d60-1b7f-4200-80e1-b518433d20b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:53:45.361941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b06d60-1b7f-4200-80e1-b518433d20b1 service swift] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:45.362224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b06d60-1b7f-4200-80e1-b518433d20b1 service swift] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:45.362502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b06d60-1b7f-4200-80e1-b518433d20b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:45.370945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b06d60-1b7f-4200-80e1-b518433d20b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:53:45.373177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b06d60-1b7f-4200-80e1-b518433d20b1 service swift] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:53:45.381247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1217/2470] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:53:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:53:55.262574 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=71 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:55.378017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=51 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:57.432823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84e8b-71f8-4a8a-886a-a8de53e2c196 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:53:57.433379 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84e8b-71f8-4a8a-886a-a8de53e2c196 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:53:57.433379 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84e8b-71f8-4a8a-886a-a8de53e2c196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:53:57.453513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-91d84e8b-71f8-4a8a-886a-a8de53e2c196 None None] MFA Rules not processed for user `b0eb3bddc1bd4557a280ad21f9092093`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:53:57.454745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-91d84e8b-71f8-4a8a-886a-a8de53e2c196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:53:57.461765 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1254/2471] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:53:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:05.533629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b94ff9-68ac-4fed-b9e8-62c4130ceff5 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:05.542035 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b94ff9-68ac-4fed-b9e8-62c4130ceff5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:05.542035 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b94ff9-68ac-4fed-b9e8-62c4130ceff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:05.558655 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c3b94ff9-68ac-4fed-b9e8-62c4130ceff5 None None] MFA Rules not processed for user `5c01a893d3ab42cc97508768791ff38e`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:05.560039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c3b94ff9-68ac-4fed-b9e8-62c4130ceff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:05.623754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1218/2472] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:54:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:06.492888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5b664-958d-47c8-8533-906d291ee674 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:06.493432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5b664-958d-47c8-8533-906d291ee674 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:06.493432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5b664-958d-47c8-8533-906d291ee674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:06.514385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ccf5b664-958d-47c8-8533-906d291ee674 None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:06.516061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ccf5b664-958d-47c8-8533-906d291ee674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:06.524465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1255/2473] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:54:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:06.541794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b94ff9-68ac-4fed-b9e8-62c4130ceff5 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:54:06.555241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b94ff9-68ac-4fed-b9e8-62c4130ceff5 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:54:06.560724 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a131665-ff4c-4f0b-9dd1-8a969af96c2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:54:06.561795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a131665-ff4c-4f0b-9dd1-8a969af96c2a service swift] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:06.562276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a131665-ff4c-4f0b-9dd1-8a969af96c2a service swift] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:06.562749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a131665-ff4c-4f0b-9dd1-8a969af96c2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:06.572817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a131665-ff4c-4f0b-9dd1-8a969af96c2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:54:06.574692 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a131665-ff4c-4f0b-9dd1-8a969af96c2a service swift] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:54:06.586274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1219/2474] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:54:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:54:16.509403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=6 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:54:16.581067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=31 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:54:40.574118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cbd9b-d5e7-4dee-9e50-6fb78ef06fe8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:40.574651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cbd9b-d5e7-4dee-9e50-6fb78ef06fe8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:40.574651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cbd9b-d5e7-4dee-9e50-6fb78ef06fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:40.596533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-f94cbd9b-d5e7-4dee-9e50-6fb78ef06fe8 None None] MFA Rules not processed for user `28436612400e421ca52f902542186a68`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:40.598313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f94cbd9b-d5e7-4dee-9e50-6fb78ef06fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:40.665970 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1256/2475] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:54:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:48.521731 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9b532-192a-4760-839d-c9a257f5effc None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:48.529501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9b532-192a-4760-839d-c9a257f5effc None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:48.529501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9b532-192a-4760-839d-c9a257f5effc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:48.543090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-00b9b532-192a-4760-839d-c9a257f5effc None None] MFA Rules not processed for user `6861433b9f4441fe842b506def255c58`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:48.544376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-00b9b532-192a-4760-839d-c9a257f5effc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:48.604779 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1220/2476] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:54:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:50.657951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=25 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:54:51.599010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cbd9b-d5e7-4dee-9e50-6fb78ef06fe8 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:54:51.659768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cbd9b-d5e7-4dee-9e50-6fb78ef06fe8 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:54:51.663556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413aeb55-3020-45d9-9fd0-44a7577221a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:54:51.664237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aeb55-3020-45d9-9fd0-44a7577221a0 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:51.664519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aeb55-3020-45d9-9fd0-44a7577221a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:51.664791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aeb55-3020-45d9-9fd0-44a7577221a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:51.674665 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413aeb55-3020-45d9-9fd0-44a7577221a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:54:51.676176 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413aeb55-3020-45d9-9fd0-44a7577221a0 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:54:51.685958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1257/2477] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:54:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:54:58.598245 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=25 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:55:01.681603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=26 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:55:06.301622 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9b532-192a-4760-839d-c9a257f5effc None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:06.311535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9b532-192a-4760-839d-c9a257f5effc None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:06.315131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d3636-db2b-42f6-9160-871c183961fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:06.315746 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3636-db2b-42f6-9160-871c183961fd service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:06.316093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3636-db2b-42f6-9160-871c183961fd service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:06.316951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3636-db2b-42f6-9160-871c183961fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:06.318774 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0b4d3636-db2b-42f6-9160-871c183961fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:06.393263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d3636-db2b-42f6-9160-871c183961fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:06.394712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d3636-db2b-42f6-9160-871c183961fd service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:06.406810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1221/2478] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:07.586404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413aeb55-3020-45d9-9fd0-44a7577221a0 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:07.588009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-413aeb55-3020-45d9-9fd0-44a7577221a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:07.599936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413aeb55-3020-45d9-9fd0-44a7577221a0 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:07.603672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707f0f1-2aba-4750-be50-dc73a5a6ec46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:07.604309 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707f0f1-2aba-4750-be50-dc73a5a6ec46 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:07.605030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707f0f1-2aba-4750-be50-dc73a5a6ec46 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:07.605030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707f0f1-2aba-4750-be50-dc73a5a6ec46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:07.606724 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a707f0f1-2aba-4750-be50-dc73a5a6ec46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:07.621113 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707f0f1-2aba-4750-be50-dc73a5a6ec46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:07.622541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707f0f1-2aba-4750-be50-dc73a5a6ec46 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:07.633661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1258/2479] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:16.401417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=34 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:55:17.117721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d3636-db2b-42f6-9160-871c183961fd service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:17.119185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0b4d3636-db2b-42f6-9160-871c183961fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:17.180506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d3636-db2b-42f6-9160-871c183961fd service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:17.184263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e5f0e-8c83-48c1-8d4d-76eb02480200 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:17.184873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e5f0e-8c83-48c1-8d4d-76eb02480200 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:17.185158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e5f0e-8c83-48c1-8d4d-76eb02480200 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:17.185446 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e5f0e-8c83-48c1-8d4d-76eb02480200 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:17.187502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e99e5f0e-8c83-48c1-8d4d-76eb02480200 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:17.201742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e5f0e-8c83-48c1-8d4d-76eb02480200 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:17.203189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e5f0e-8c83-48c1-8d4d-76eb02480200 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:17.213800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1222/2480] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:17.628723 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=6 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:55:27.209081 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=26 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:55:30.843967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8dca9-979b-4b3b-a994-9d08ad8e5477 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:30.844549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8dca9-979b-4b3b-a994-9d08ad8e5477 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:30.844549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8dca9-979b-4b3b-a994-9d08ad8e5477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:30.865219 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ecb8dca9-979b-4b3b-a994-9d08ad8e5477 None None] MFA Rules not processed for user `9348310fd06b473786d1c6be4f20c481`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:55:30.866457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ecb8dca9-979b-4b3b-a994-9d08ad8e5477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:30.926356 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1259/2481] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:55:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:55:31.253263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e5f0e-8c83-48c1-8d4d-76eb02480200 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:31.262898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e5f0e-8c83-48c1-8d4d-76eb02480200 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:31.266553 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de999c7c-5bf5-400d-9165-68e8f5526841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:31.267150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de999c7c-5bf5-400d-9165-68e8f5526841 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:31.267435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de999c7c-5bf5-400d-9165-68e8f5526841 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:31.267710 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de999c7c-5bf5-400d-9165-68e8f5526841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:31.269477 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-de999c7c-5bf5-400d-9165-68e8f5526841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:31.280548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de999c7c-5bf5-400d-9165-68e8f5526841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:31.281983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de999c7c-5bf5-400d-9165-68e8f5526841 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:31.293442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1223/2482] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:31.730495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8dca9-979b-4b3b-a994-9d08ad8e5477 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:31.732269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ecb8dca9-979b-4b3b-a994-9d08ad8e5477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:31.811820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8dca9-979b-4b3b-a994-9d08ad8e5477 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:31.817492 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bd228-cd63-4475-a31e-8272973b9f74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:31.818128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bd228-cd63-4475-a31e-8272973b9f74 service placement] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:31.818464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bd228-cd63-4475-a31e-8272973b9f74 service placement] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:31.818745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bd228-cd63-4475-a31e-8272973b9f74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:31.820593 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0f6bd228-cd63-4475-a31e-8272973b9f74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:31.833625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bd228-cd63-4475-a31e-8272973b9f74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:31.835275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bd228-cd63-4475-a31e-8272973b9f74 service placement] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:31.845813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1260/2483] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:37.261389 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de999c7c-5bf5-400d-9165-68e8f5526841 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:37.270954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de999c7c-5bf5-400d-9165-68e8f5526841 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:37.275173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9b96-0d6e-4ea3-82fd-1b7a5e18d71f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:37.276100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9b96-0d6e-4ea3-82fd-1b7a5e18d71f service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:37.276602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9b96-0d6e-4ea3-82fd-1b7a5e18d71f service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:37.277099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9b96-0d6e-4ea3-82fd-1b7a5e18d71f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:37.279158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-bf2d9b96-0d6e-4ea3-82fd-1b7a5e18d71f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:37.290737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d9b96-0d6e-4ea3-82fd-1b7a5e18d71f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:37.292565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d9b96-0d6e-4ea3-82fd-1b7a5e18d71f service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:37.302252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1224/2484] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:38.295907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bd228-cd63-4475-a31e-8272973b9f74 service placement] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:38.297339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0f6bd228-cd63-4475-a31e-8272973b9f74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:38.325067 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bd228-cd63-4475-a31e-8272973b9f74 service placement] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:38.333313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728252f1-37af-4e37-a8ef-2adb9c6ef34f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:38.334344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728252f1-37af-4e37-a8ef-2adb9c6ef34f service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:38.334848 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728252f1-37af-4e37-a8ef-2adb9c6ef34f service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:38.335221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728252f1-37af-4e37-a8ef-2adb9c6ef34f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:38.340993 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-728252f1-37af-4e37-a8ef-2adb9c6ef34f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:38.386455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728252f1-37af-4e37-a8ef-2adb9c6ef34f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:38.388283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728252f1-37af-4e37-a8ef-2adb9c6ef34f service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:38.414200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1261/2485] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:47.297867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=12 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:55:48.397564 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=63 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:55:50.411038 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9b96-0d6e-4ea3-82fd-1b7a5e18d71f service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:50.412620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-bf2d9b96-0d6e-4ea3-82fd-1b7a5e18d71f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:50.430434 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9b96-0d6e-4ea3-82fd-1b7a5e18d71f service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:50.437257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76278aa0-3222-456b-b9ca-c252a63e397d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:50.437884 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278aa0-3222-456b-b9ca-c252a63e397d service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:50.438220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278aa0-3222-456b-b9ca-c252a63e397d service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:50.438675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278aa0-3222-456b-b9ca-c252a63e397d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:50.440785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-76278aa0-3222-456b-b9ca-c252a63e397d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:50.460472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76278aa0-3222-456b-b9ca-c252a63e397d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:50.462090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76278aa0-3222-456b-b9ca-c252a63e397d service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:50.484858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1225/2486] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:53.520908 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728252f1-37af-4e37-a8ef-2adb9c6ef34f service nova] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:53.534670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728252f1-37af-4e37-a8ef-2adb9c6ef34f service nova] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:53.540582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178140b-8521-46cb-a08a-cc610f68e11b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:53.541558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178140b-8521-46cb-a08a-cc610f68e11b service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:53.541856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178140b-8521-46cb-a08a-cc610f68e11b service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:53.542135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178140b-8521-46cb-a08a-cc610f68e11b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:53.544525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e178140b-8521-46cb-a08a-cc610f68e11b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:53.884016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178140b-8521-46cb-a08a-cc610f68e11b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:53.884016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178140b-8521-46cb-a08a-cc610f68e11b service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:53.884016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1262/2487] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:54.404775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f6fc6-75f0-4f3c-9ebf-6d97b1895bcd None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:54.404775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f6fc6-75f0-4f3c-9ebf-6d97b1895bcd None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:54.405279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f6fc6-75f0-4f3c-9ebf-6d97b1895bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:54.426382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-df7f6fc6-75f0-4f3c-9ebf-6d97b1895bcd None None] MFA Rules not processed for user `436adf49f3e24e03971017ea6dd33ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:55:54.428080 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-df7f6fc6-75f0-4f3c-9ebf-6d97b1895bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:54.491263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1226/2488] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:55:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:55:55.643728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178140b-8521-46cb-a08a-cc610f68e11b service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:55.660034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178140b-8521-46cb-a08a-cc610f68e11b service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:55.677607 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceed7ca-db95-4817-96b9-66b9ff5c0c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:55.678349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceed7ca-db95-4817-96b9-66b9ff5c0c6f service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:55.678729 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceed7ca-db95-4817-96b9-66b9ff5c0c6f service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:55.679252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceed7ca-db95-4817-96b9-66b9ff5c0c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:55.681367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8ceed7ca-db95-4817-96b9-66b9ff5c0c6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:55.697682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceed7ca-db95-4817-96b9-66b9ff5c0c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:55.700176 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceed7ca-db95-4817-96b9-66b9ff5c0c6f service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:55.714219 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1263/2489] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:57.829040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f6fc6-75f0-4f3c-9ebf-6d97b1895bcd None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:57.842732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f6fc6-75f0-4f3c-9ebf-6d97b1895bcd None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:57.848681 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf9035-9da8-4e0a-a9fd-d74c2a12318b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:57.849316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf9035-9da8-4e0a-a9fd-d74c2a12318b service placement] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:57.849633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf9035-9da8-4e0a-a9fd-d74c2a12318b service placement] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:57.850021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf9035-9da8-4e0a-a9fd-d74c2a12318b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:57.852109 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0aaf9035-9da8-4e0a-a9fd-d74c2a12318b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:57.875057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf9035-9da8-4e0a-a9fd-d74c2a12318b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:57.877550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf9035-9da8-4e0a-a9fd-d74c2a12318b service placement] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:57.893436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1227/2490] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:57.993530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceed7ca-db95-4817-96b9-66b9ff5c0c6f service nova] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:57.995473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8ceed7ca-db95-4817-96b9-66b9ff5c0c6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:58.009992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceed7ca-db95-4817-96b9-66b9ff5c0c6f service nova] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:58.014658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfe578-bc7a-40f9-9e8e-8cfc2a5f288f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:58.015640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfe578-bc7a-40f9-9e8e-8cfc2a5f288f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:58.016098 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfe578-bc7a-40f9-9e8e-8cfc2a5f288f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:58.016544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfe578-bc7a-40f9-9e8e-8cfc2a5f288f admin admin] PATH_INFO: `/v3/users/5c01a893d3ab42cc97508768791ff38e` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:58.018534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfe578-bc7a-40f9-9e8e-8cfc2a5f288f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c01a893d3ab42cc97508768791ff38e)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:58.021062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfe578-bc7a-40f9-9e8e-8cfc2a5f288f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:58.240747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a3dfe578-bc7a-40f9-9e8e-8cfc2a5f288f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c01a893d3ab42cc97508768791ff38e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:55:58.253832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a3dfe578-bc7a-40f9-9e8e-8cfc2a5f288f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c01a893d3ab42cc97508768791ff38e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:55:58.254569 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a3dfe578-bc7a-40f9-9e8e-8cfc2a5f288f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c01a893d3ab42cc97508768791ff38e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:55:58.255179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a3dfe578-bc7a-40f9-9e8e-8cfc2a5f288f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c01a893d3ab42cc97508768791ff38e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:55:58.333617 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a3dfe578-bc7a-40f9-9e8e-8cfc2a5f288f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c01a893d3ab42cc97508768791ff38e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:55:58.346156 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1264/2491] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:55:57 2022] DELETE /identity/v3/users/5c01a893d3ab42cc97508768791ff38e => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:55:58.604826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf9035-9da8-4e0a-a9fd-d74c2a12318b service placement] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:58.639749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf9035-9da8-4e0a-a9fd-d74c2a12318b service placement] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:58.730175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:58.730938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:58.731229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:58.731655 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] PATH_INFO: `/v3/projects/853df33daae14bd9825788afd555e11a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:58.732788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=853df33daae14bd9825788afd555e11a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:58.734774 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:58.847193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '853df33daae14bd9825788afd555e11a', 'request_id': 'req-bef18cb5-1e69-4212-9088-7ca97a2c9d80'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:55:58.857558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '853df33daae14bd9825788afd555e11a', 'request_id': 'req-bef18cb5-1e69-4212-9088-7ca97a2c9d80'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:55:58.871974 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '853df33daae14bd9825788afd555e11a', 'request_id': 'req-bef18cb5-1e69-4212-9088-7ca97a2c9d80'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:55:58.970158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] The token cache is being invalidate because project 853df33daae14bd9825788afd555e11a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:55:58.970524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:55:58.971455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bef18cb5-1e69-4212-9088-7ca97a2c9d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:55:58.973268 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1228/2492] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:55:58 2022] DELETE /identity/v3/projects/853df33daae14bd9825788afd555e11a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:55:58.979850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69386c-3145-4296-a791-2d7558476536 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:58.979850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69386c-3145-4296-a791-2d7558476536 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:58.979850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69386c-3145-4296-a791-2d7558476536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:59.007095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-1d69386c-3145-4296-a791-2d7558476536 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:55:59.007701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1d69386c-3145-4296-a791-2d7558476536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:59.075732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:55:59.090788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1265/2493] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:55:59.099703 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1425644-3700-473b-96e6-6b277626acb5 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:59.099939 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1425644-3700-473b-96e6-6b277626acb5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:59.100213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1425644-3700-473b-96e6-6b277626acb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:59.121095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c1425644-3700-473b-96e6-6b277626acb5 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:55:59.122352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c1425644-3700-473b-96e6-6b277626acb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:59.131509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1229/2494] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:55:59.136028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69386c-3145-4296-a791-2d7558476536 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:59.146674 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69386c-3145-4296-a791-2d7558476536 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:59.152237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af72f9-f33c-46c5-b6a1-7092376c2801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:59.152862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af72f9-f33c-46c5-b6a1-7092376c2801 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:59.153149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af72f9-f33c-46c5-b6a1-7092376c2801 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:59.153432 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af72f9-f33c-46c5-b6a1-7092376c2801 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:59.154098 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af72f9-f33c-46c5-b6a1-7092376c2801 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:55:59.154409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af72f9-f33c-46c5-b6a1-7092376c2801 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:59.156143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af72f9-f33c-46c5-b6a1-7092376c2801 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:59.170413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1266/2495] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:55:59.174503 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1425644-3700-473b-96e6-6b277626acb5 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:59.179840 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:55:59.186990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1425644-3700-473b-96e6-6b277626acb5 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:59.193208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f19d9-198e-4ef8-b267-25cffbc04407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:59.193851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f19d9-198e-4ef8-b267-25cffbc04407 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:59.194426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f19d9-198e-4ef8-b267-25cffbc04407 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:59.194917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f19d9-198e-4ef8-b267-25cffbc04407 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:59.195676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f19d9-198e-4ef8-b267-25cffbc04407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:59.198715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f19d9-198e-4ef8-b267-25cffbc04407 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:59.322783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1230/2496] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:55:59 2022] POST /identity/v3/projects => generated 389 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:55:59.328340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af72f9-f33c-46c5-b6a1-7092376c2801 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:59.341228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af72f9-f33c-46c5-b6a1-7092376c2801 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:59.439916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da691c64-0e85-4300-a330-f7086eb29d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:59.440552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691c64-0e85-4300-a330-f7086eb29d46 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:59.440858 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691c64-0e85-4300-a330-f7086eb29d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:59.441124 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691c64-0e85-4300-a330-f7086eb29d46 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:59.441768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da691c64-0e85-4300-a330-f7086eb29d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:59.443818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da691c64-0e85-4300-a330-f7086eb29d46 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:59.582719 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1267/2497] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:55:59 2022] POST /identity/v3/users => generated 295 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:55:59.584919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f19d9-198e-4ef8-b267-25cffbc04407 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:59.604790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f19d9-198e-4ef8-b267-25cffbc04407 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:59.617387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a4f33-abf0-4721-bd52-58874d581afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:59.618442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a4f33-abf0-4721-bd52-58874d581afc admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:59.618935 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a4f33-abf0-4721-bd52-58874d581afc admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:59.619436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a4f33-abf0-4721-bd52-58874d581afc admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:59.620686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a4f33-abf0-4721-bd52-58874d581afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:59.623884 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a4f33-abf0-4721-bd52-58874d581afc admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:59.648321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1231/2498] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:55:59.649486 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da691c64-0e85-4300-a330-f7086eb29d46 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:59.661531 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da691c64-0e85-4300-a330-f7086eb29d46 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:59.668034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1ea71-d04d-443f-b0ca-2e6751b0c931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:59.668752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1ea71-d04d-443f-b0ca-2e6751b0c931 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:59.669078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1ea71-d04d-443f-b0ca-2e6751b0c931 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:59.673964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1ea71-d04d-443f-b0ca-2e6751b0c931 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:59.673964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1ea71-d04d-443f-b0ca-2e6751b0c931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:59.673964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1ea71-d04d-443f-b0ca-2e6751b0c931 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:59.687434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1268/2499] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:55:59.692172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a4f33-abf0-4721-bd52-58874d581afc admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:59.704499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a4f33-abf0-4721-bd52-58874d581afc admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:59.710088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f10a45-e435-46cd-af16-5796b5851d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:59.710908 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10a45-e435-46cd-af16-5796b5851d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:59.711485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10a45-e435-46cd-af16-5796b5851d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:59.712020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10a45-e435-46cd-af16-5796b5851d57 admin admin] PATH_INFO: `/v3/projects/10b0a72955244448954be539f99435e9/users/91966686fa7f4c5bb326cdb1e2abc1d9/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:59.742273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f10a45-e435-46cd-af16-5796b5851d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b0a72955244448954be539f99435e9, user_id=91966686fa7f4c5bb326cdb1e2abc1d9, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:59.745428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f10a45-e435-46cd-af16-5796b5851d57 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:59.814968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1232/2500] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:55:59 2022] PUT /identity/v3/projects/10b0a72955244448954be539f99435e9/users/91966686fa7f4c5bb326cdb1e2abc1d9/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:55:59.820806 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a628e29-be36-451e-b549-32367cbb48d5 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:59.821126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a628e29-be36-451e-b549-32367cbb48d5 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:59.821367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a628e29-be36-451e-b549-32367cbb48d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:59.849568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3a628e29-be36-451e-b549-32367cbb48d5 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:55:59.849568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3a628e29-be36-451e-b549-32367cbb48d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:59.909008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1269/2501] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:55:59.919540 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f10a45-e435-46cd-af16-5796b5851d57 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:59.921437 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-15f10a45-e435-46cd-af16-5796b5851d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:59.994747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f10a45-e435-46cd-af16-5796b5851d57 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:59.999147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84decca-876a-4c2b-b49b-384e5d30412b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:59.999813 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84decca-876a-4c2b-b49b-384e5d30412b service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:00.000116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84decca-876a-4c2b-b49b-384e5d30412b service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:00.000410 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84decca-876a-4c2b-b49b-384e5d30412b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:00.012867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84decca-876a-4c2b-b49b-384e5d30412b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:00.015853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84decca-876a-4c2b-b49b-384e5d30412b service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:00.039186 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1233/2502] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:00.509102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a628e29-be36-451e-b549-32367cbb48d5 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:00.511605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3a628e29-be36-451e-b549-32367cbb48d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:00.526782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a628e29-be36-451e-b549-32367cbb48d5 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:00.549648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900403a6-52c1-419b-9ee8-8f0807bd9f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:00.551138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900403a6-52c1-419b-9ee8-8f0807bd9f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:00.551436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900403a6-52c1-419b-9ee8-8f0807bd9f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:00.552566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900403a6-52c1-419b-9ee8-8f0807bd9f0d admin admin] PATH_INFO: `/v3/users/28436612400e421ca52f902542186a68` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:00.555511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900403a6-52c1-419b-9ee8-8f0807bd9f0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=28436612400e421ca52f902542186a68)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:00.557718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900403a6-52c1-419b-9ee8-8f0807bd9f0d admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:00.884184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-900403a6-52c1-419b-9ee8-8f0807bd9f0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28436612400e421ca52f902542186a68'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:00.896809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:00.906577 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-900403a6-52c1-419b-9ee8-8f0807bd9f0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28436612400e421ca52f902542186a68'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:00.908232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-900403a6-52c1-419b-9ee8-8f0807bd9f0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28436612400e421ca52f902542186a68'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:00.908938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-900403a6-52c1-419b-9ee8-8f0807bd9f0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28436612400e421ca52f902542186a68'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:01.007321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-900403a6-52c1-419b-9ee8-8f0807bd9f0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28436612400e421ca52f902542186a68'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:01.024686 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1270/2503] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:56:00 2022] DELETE /identity/v3/users/28436612400e421ca52f902542186a68 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:01.489653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84decca-876a-4c2b-b49b-384e5d30412b service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:01.501297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84decca-876a-4c2b-b49b-384e5d30412b service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:01.592128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:01.592785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:01.593102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:01.593388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] PATH_INFO: `/v3/projects/7e4bab21dc3141bd89ee2ea54fe0c14c` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:01.594566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e4bab21dc3141bd89ee2ea54fe0c14c)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:01.596545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:01.798511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4bab21dc3141bd89ee2ea54fe0c14c', 'request_id': 'req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:01.800361 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:01.808849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4bab21dc3141bd89ee2ea54fe0c14c', 'request_id': 'req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:01.817405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4bab21dc3141bd89ee2ea54fe0c14c', 'request_id': 'req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:01.917675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] The token cache is being invalidate because project 7e4bab21dc3141bd89ee2ea54fe0c14c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:56:01.918311 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:01.919749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:01.922160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1234/2504] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:56:01 2022] DELETE /identity/v3/projects/7e4bab21dc3141bd89ee2ea54fe0c14c => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:06.267673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264b53e-abf7-4804-b2f5-9be23684c3c1 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:06.268259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264b53e-abf7-4804-b2f5-9be23684c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:06.268378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264b53e-abf7-4804-b2f5-9be23684c3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:06.300549 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3264b53e-abf7-4804-b2f5-9be23684c3c1 None None] MFA Rules not processed for user `91966686fa7f4c5bb326cdb1e2abc1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:06.301846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3264b53e-abf7-4804-b2f5-9be23684c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:06.403198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1271/2505] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:06.412168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:06.414573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:06.498782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2b7ed-0a0e-4bca-881f-d9ff7f3dce4f admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:06.504430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e169c39-8204-4273-a383-4be6dbd6b799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:06.505651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169c39-8204-4273-a383-4be6dbd6b799 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:06.505938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169c39-8204-4273-a383-4be6dbd6b799 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:06.506219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169c39-8204-4273-a383-4be6dbd6b799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:06.516515 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e169c39-8204-4273-a383-4be6dbd6b799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:06.517660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e169c39-8204-4273-a383-4be6dbd6b799 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:06.529209 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1235/2506] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:09.277199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaf348-4e8d-4bb5-a6d8-7829f321ef2d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:09.277707 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaf348-4e8d-4bb5-a6d8-7829f321ef2d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:09.277851 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaf348-4e8d-4bb5-a6d8-7829f321ef2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:09.299251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9efaf348-4e8d-4bb5-a6d8-7829f321ef2d None None] MFA Rules not processed for user `91966686fa7f4c5bb326cdb1e2abc1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:09.300771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9efaf348-4e8d-4bb5-a6d8-7829f321ef2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:09.309951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1272/2507] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:16.524399 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:DELETE=3,keystone:SELECT=40,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:17.396602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e169c39-8204-4273-a383-4be6dbd6b799 service nova] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:17.398287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5e169c39-8204-4273-a383-4be6dbd6b799 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:17.413981 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e169c39-8204-4273-a383-4be6dbd6b799 service nova] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:17.419601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6973655-c38d-469b-ad0e-de56b46e9920 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:17.420257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6973655-c38d-469b-ad0e-de56b46e9920 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:17.424169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6973655-c38d-469b-ad0e-de56b46e9920 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:17.424169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6973655-c38d-469b-ad0e-de56b46e9920 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:17.424169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a6973655-c38d-469b-ad0e-de56b46e9920 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:17.435837 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6973655-c38d-469b-ad0e-de56b46e9920 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:17.437477 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6973655-c38d-469b-ad0e-de56b46e9920 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:17.450197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1236/2508] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:19.293561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=45,keystone:DELETE=2,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:27.444967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=6 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:36.469124 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efaf348-4e8d-4bb5-a6d8-7829f321ef2d None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:36.471557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9efaf348-4e8d-4bb5-a6d8-7829f321ef2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:36.577203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efaf348-4e8d-4bb5-a6d8-7829f321ef2d None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:36.582805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d45ee0-c90e-4059-a6b7-55871ba425c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:36.584341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d45ee0-c90e-4059-a6b7-55871ba425c1 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:36.584743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d45ee0-c90e-4059-a6b7-55871ba425c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:36.585132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d45ee0-c90e-4059-a6b7-55871ba425c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:36.587992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f1d45ee0-c90e-4059-a6b7-55871ba425c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:36.696302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d45ee0-c90e-4059-a6b7-55871ba425c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:36.698149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d45ee0-c90e-4059-a6b7-55871ba425c1 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:36.715107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1273/2509] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:42.113267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6973655-c38d-469b-ad0e-de56b46e9920 service nova] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:42.114744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a6973655-c38d-469b-ad0e-de56b46e9920 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:42.129039 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6973655-c38d-469b-ad0e-de56b46e9920 service nova] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:42.134174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a244e-c758-4505-b126-54ce528df240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:42.134812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a244e-c758-4505-b126-54ce528df240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:42.135100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a244e-c758-4505-b126-54ce528df240 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:42.135385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a244e-c758-4505-b126-54ce528df240 admin admin] PATH_INFO: `/v3/users/6861433b9f4441fe842b506def255c58` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:42.136536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a244e-c758-4505-b126-54ce528df240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6861433b9f4441fe842b506def255c58)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:42.138456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a244e-c758-4505-b126-54ce528df240 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:42.340042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-480a244e-c758-4505-b126-54ce528df240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6861433b9f4441fe842b506def255c58'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:42.340848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-480a244e-c758-4505-b126-54ce528df240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6861433b9f4441fe842b506def255c58'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:42.354792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-480a244e-c758-4505-b126-54ce528df240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6861433b9f4441fe842b506def255c58'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:42.468148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-480a244e-c758-4505-b126-54ce528df240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6861433b9f4441fe842b506def255c58'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:42.476552 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-480a244e-c758-4505-b126-54ce528df240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6861433b9f4441fe842b506def255c58'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:42.482890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1237/2510] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:56:42 2022] DELETE /identity/v3/users/6861433b9f4441fe842b506def255c58 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:42.818825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d45ee0-c90e-4059-a6b7-55871ba425c1 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:42.833745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d45ee0-c90e-4059-a6b7-55871ba425c1 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:42.900012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:42.900785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:42.901116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:42.901471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] PATH_INFO: `/v3/projects/bc372e59524442debba38c57df458071` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:42.902876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc372e59524442debba38c57df458071)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:42.905336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:43.028052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc372e59524442debba38c57df458071', 'request_id': 'req-d0590e1a-9a5c-48df-a3db-2378b6af42a4'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:43.030697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:43.128988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc372e59524442debba38c57df458071', 'request_id': 'req-d0590e1a-9a5c-48df-a3db-2378b6af42a4'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:43.138236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc372e59524442debba38c57df458071', 'request_id': 'req-d0590e1a-9a5c-48df-a3db-2378b6af42a4'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:43.150690 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] The token cache is being invalidate because project bc372e59524442debba38c57df458071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:56:43.150963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:43.151638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d0590e1a-9a5c-48df-a3db-2378b6af42a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:43.153281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1274/2511] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:56:42 2022] DELETE /identity/v3/projects/bc372e59524442debba38c57df458071 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:43.158987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06cfca-d3f5-4d85-8304-b69d0f2d8511 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:43.159500 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06cfca-d3f5-4d85-8304-b69d0f2d8511 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:43.159594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06cfca-d3f5-4d85-8304-b69d0f2d8511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:43.180486 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ff06cfca-d3f5-4d85-8304-b69d0f2d8511 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:43.181719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ff06cfca-d3f5-4d85-8304-b69d0f2d8511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:43.242555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1238/2512] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:43.250893 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7354c-31f3-45d0-b317-362e99cf79c7 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:43.251181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7354c-31f3-45d0-b317-362e99cf79c7 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:43.251460 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7354c-31f3-45d0-b317-362e99cf79c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:43.272331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0bc7354c-31f3-45d0-b317-362e99cf79c7 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:43.273569 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0bc7354c-31f3-45d0-b317-362e99cf79c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:43.281747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1275/2513] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:43.286370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06cfca-d3f5-4d85-8304-b69d0f2d8511 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:43.308706 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06cfca-d3f5-4d85-8304-b69d0f2d8511 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:43.315440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845d04e-c78f-4d61-893d-947ee86af1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:43.316428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845d04e-c78f-4d61-893d-947ee86af1fc admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:43.316900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845d04e-c78f-4d61-893d-947ee86af1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:43.317363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845d04e-c78f-4d61-893d-947ee86af1fc admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:43.318698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845d04e-c78f-4d61-893d-947ee86af1fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:56:43.319442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845d04e-c78f-4d61-893d-947ee86af1fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:43.322065 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845d04e-c78f-4d61-893d-947ee86af1fc admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:43.340800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1239/2514] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:56:43.345227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7354c-31f3-45d0-b317-362e99cf79c7 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:43.355537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7354c-31f3-45d0-b317-362e99cf79c7 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:43.360376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7e1c4-14e0-48a4-9c30-d70cfc69dd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:43.361507 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e1c4-14e0-48a4-9c30-d70cfc69dd25 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:43.361723 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e1c4-14e0-48a4-9c30-d70cfc69dd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:43.361999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e1c4-14e0-48a4-9c30-d70cfc69dd25 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:43.362623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7e1c4-14e0-48a4-9c30-d70cfc69dd25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:43.364591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7e1c4-14e0-48a4-9c30-d70cfc69dd25 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:43.465103 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1276/2515] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:43 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:43.471153 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845d04e-c78f-4d61-893d-947ee86af1fc admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:43.482557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845d04e-c78f-4d61-893d-947ee86af1fc admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:43.541026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca960e-a846-497e-b598-7be9ea03fc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:43.541559 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca960e-a846-497e-b598-7be9ea03fc95 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:43.541846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca960e-a846-497e-b598-7be9ea03fc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:43.542150 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca960e-a846-497e-b598-7be9ea03fc95 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:43.542808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca960e-a846-497e-b598-7be9ea03fc95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:43.544764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca960e-a846-497e-b598-7be9ea03fc95 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:43.553049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:43.654408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1240/2516] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:43 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:43.658653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7e1c4-14e0-48a4-9c30-d70cfc69dd25 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:43.670401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7e1c4-14e0-48a4-9c30-d70cfc69dd25 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:43.675891 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cda23-e7d1-48aa-b7a4-60a96094d033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:43.676508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cda23-e7d1-48aa-b7a4-60a96094d033 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:43.680875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cda23-e7d1-48aa-b7a4-60a96094d033 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:43.680875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cda23-e7d1-48aa-b7a4-60a96094d033 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:43.680875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0cda23-e7d1-48aa-b7a4-60a96094d033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:43.680875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0cda23-e7d1-48aa-b7a4-60a96094d033 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:43.691308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1277/2517] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:43.696586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca960e-a846-497e-b598-7be9ea03fc95 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:43.707967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca960e-a846-497e-b598-7be9ea03fc95 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:43.714615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db69ca3-4452-462f-bd8b-1c4fd226939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:43.715281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db69ca3-4452-462f-bd8b-1c4fd226939c admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:43.715569 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db69ca3-4452-462f-bd8b-1c4fd226939c admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:43.724577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db69ca3-4452-462f-bd8b-1c4fd226939c admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:43.724577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db69ca3-4452-462f-bd8b-1c4fd226939c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:43.724577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db69ca3-4452-462f-bd8b-1c4fd226939c admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:43.731032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1241/2518] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:43.735761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cda23-e7d1-48aa-b7a4-60a96094d033 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:43.746978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cda23-e7d1-48aa-b7a4-60a96094d033 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:43.752022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b22e9c-2ccb-4966-9a60-7a2e29a9f727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:43.761305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b22e9c-2ccb-4966-9a60-7a2e29a9f727 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:43.761305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b22e9c-2ccb-4966-9a60-7a2e29a9f727 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:43.761305 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b22e9c-2ccb-4966-9a60-7a2e29a9f727 admin admin] PATH_INFO: `/v3/projects/1b6d6b93ae5e4ee6b0893b59a2f98136/users/bcf7dbeb06404ed39e537d437cb2bb55/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:43.770160 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b22e9c-2ccb-4966-9a60-7a2e29a9f727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b6d6b93ae5e4ee6b0893b59a2f98136, user_id=bcf7dbeb06404ed39e537d437cb2bb55, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:43.773517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b22e9c-2ccb-4966-9a60-7a2e29a9f727 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:43.902870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1278/2519] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:43 2022] PUT /identity/v3/projects/1b6d6b93ae5e4ee6b0893b59a2f98136/users/bcf7dbeb06404ed39e537d437cb2bb55/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:43.915289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41607fa7-bc4a-4b57-bf01-34d9a744f6b0 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:43.915674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41607fa7-bc4a-4b57-bf01-34d9a744f6b0 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:43.915979 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41607fa7-bc4a-4b57-bf01-34d9a744f6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:43.947021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-41607fa7-bc4a-4b57-bf01-34d9a744f6b0 None None] MFA Rules not processed for user `bcf7dbeb06404ed39e537d437cb2bb55`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:43.948674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-41607fa7-bc4a-4b57-bf01-34d9a744f6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:44.031671 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1242/2520] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:44.036692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b22e9c-2ccb-4966-9a60-7a2e29a9f727 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:44.102064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b22e9c-2ccb-4966-9a60-7a2e29a9f727 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:44.107693 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f36fa-afa5-4c78-840f-196f38e78813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:44.108358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f36fa-afa5-4c78-840f-196f38e78813 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:44.108712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f36fa-afa5-4c78-840f-196f38e78813 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:44.108927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f36fa-afa5-4c78-840f-196f38e78813 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:44.109585 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f36fa-afa5-4c78-840f-196f38e78813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:44.111561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f36fa-afa5-4c78-840f-196f38e78813 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:44.218353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1279/2521] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:44 2022] POST /identity/v3/projects => generated 369 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:44.222573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41607fa7-bc4a-4b57-bf01-34d9a744f6b0 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:44.233454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41607fa7-bc4a-4b57-bf01-34d9a744f6b0 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:44.290752 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9b065-a9a5-480f-be94-a3dcb96b140e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:44.291374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9b065-a9a5-480f-be94-a3dcb96b140e admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:44.291659 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9b065-a9a5-480f-be94-a3dcb96b140e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:44.291968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9b065-a9a5-480f-be94-a3dcb96b140e admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:44.292602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9b065-a9a5-480f-be94-a3dcb96b140e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:44.294546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9b065-a9a5-480f-be94-a3dcb96b140e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:44.400404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1243/2522] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:44 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:44.404888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f36fa-afa5-4c78-840f-196f38e78813 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:44.415734 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f36fa-afa5-4c78-840f-196f38e78813 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:44.420489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197b346-a9a0-40da-8fd3-5cdc5c1885c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:44.421162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197b346-a9a0-40da-8fd3-5cdc5c1885c0 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:44.421448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197b346-a9a0-40da-8fd3-5cdc5c1885c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:44.421740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197b346-a9a0-40da-8fd3-5cdc5c1885c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:44.422389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197b346-a9a0-40da-8fd3-5cdc5c1885c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:44.424039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197b346-a9a0-40da-8fd3-5cdc5c1885c0 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:44.435299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1280/2523] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:44.439843 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9b065-a9a5-480f-be94-a3dcb96b140e admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:44.451765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9b065-a9a5-480f-be94-a3dcb96b140e admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:44.456438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1895d-12d6-41e8-9bcf-57f2c9c5ae16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:44.457107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1895d-12d6-41e8-9bcf-57f2c9c5ae16 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:44.457392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1895d-12d6-41e8-9bcf-57f2c9c5ae16 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:44.457674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1895d-12d6-41e8-9bcf-57f2c9c5ae16 admin admin] PATH_INFO: `/v3/projects/e70bb23e956d4893a3e22092df6a7b03/users/95beaa3b8bd04e1eb0558445b8819d39/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:44.473892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1895d-12d6-41e8-9bcf-57f2c9c5ae16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70bb23e956d4893a3e22092df6a7b03, user_id=95beaa3b8bd04e1eb0558445b8819d39, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:44.476593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1895d-12d6-41e8-9bcf-57f2c9c5ae16 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:44.576622 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1244/2524] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:44 2022] PUT /identity/v3/projects/e70bb23e956d4893a3e22092df6a7b03/users/95beaa3b8bd04e1eb0558445b8819d39/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:44.580224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197b346-a9a0-40da-8fd3-5cdc5c1885c0 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:44.591335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197b346-a9a0-40da-8fd3-5cdc5c1885c0 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:44.646902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:44.654832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8273ca-7e45-4d9f-a52f-df3a947a5d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:44.655462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8273ca-7e45-4d9f-a52f-df3a947a5d11 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:44.655808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8273ca-7e45-4d9f-a52f-df3a947a5d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:44.656179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8273ca-7e45-4d9f-a52f-df3a947a5d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:44.656835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8273ca-7e45-4d9f-a52f-df3a947a5d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:44.658447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8273ca-7e45-4d9f-a52f-df3a947a5d11 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:44.671706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1281/2525] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:44.676050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1895d-12d6-41e8-9bcf-57f2c9c5ae16 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:44.687778 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1895d-12d6-41e8-9bcf-57f2c9c5ae16 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:44.692811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aef7e9-850a-4fb9-a91b-2ae01e9a564e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:44.694155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aef7e9-850a-4fb9-a91b-2ae01e9a564e admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:44.694504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aef7e9-850a-4fb9-a91b-2ae01e9a564e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:44.695068 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aef7e9-850a-4fb9-a91b-2ae01e9a564e admin admin] PATH_INFO: `/v3/projects/e70bb23e956d4893a3e22092df6a7b03/users/95beaa3b8bd04e1eb0558445b8819d39/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:44.697103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aef7e9-850a-4fb9-a91b-2ae01e9a564e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70bb23e956d4893a3e22092df6a7b03, user_id=95beaa3b8bd04e1eb0558445b8819d39, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:44.699789 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aef7e9-850a-4fb9-a91b-2ae01e9a564e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:44.712077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1245/2526] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:44 2022] PUT /identity/v3/projects/e70bb23e956d4893a3e22092df6a7b03/users/95beaa3b8bd04e1eb0558445b8819d39/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:56:44.726035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2047ce-327f-4847-b681-8ecb14dc627f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:44.726328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2047ce-327f-4847-b681-8ecb14dc627f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:44.726610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2047ce-327f-4847-b681-8ecb14dc627f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:44.757902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0a2047ce-327f-4847-b681-8ecb14dc627f None None] MFA Rules not processed for user `95beaa3b8bd04e1eb0558445b8819d39`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:44.760019 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0a2047ce-327f-4847-b681-8ecb14dc627f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:44.849477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1282/2527] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:44.860816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aef7e9-850a-4fb9-a91b-2ae01e9a564e admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:44.863075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-19aef7e9-850a-4fb9-a91b-2ae01e9a564e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:44.892554 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:44.935002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aef7e9-850a-4fb9-a91b-2ae01e9a564e admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:44.939499 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a7e6a-f6b6-42ad-b034-1e8e19c5764e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:44.940363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7e6a-f6b6-42ad-b034-1e8e19c5764e service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:44.940650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7e6a-f6b6-42ad-b034-1e8e19c5764e service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:44.940935 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7e6a-f6b6-42ad-b034-1e8e19c5764e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:44.951695 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a7e6a-f6b6-42ad-b034-1e8e19c5764e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:44.953197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a7e6a-f6b6-42ad-b034-1e8e19c5764e service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:44.964785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1246/2528] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:45.086779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2047ce-327f-4847-b681-8ecb14dc627f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:45.089170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0a2047ce-327f-4847-b681-8ecb14dc627f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:45.102481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2047ce-327f-4847-b681-8ecb14dc627f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:45.110214 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a55645-6020-4367-8641-132caa65b5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:45.110975 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a55645-6020-4367-8641-132caa65b5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:45.111923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a55645-6020-4367-8641-132caa65b5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:45.111923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a55645-6020-4367-8641-132caa65b5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:45.181603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a55645-6020-4367-8641-132caa65b5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:45.183075 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a55645-6020-4367-8641-132caa65b5a3 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:45.193988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1283/2529] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:48.671570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33aacf-bebd-4119-9361-1a4fed1af776 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:48.672323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33aacf-bebd-4119-9361-1a4fed1af776 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:48.672461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33aacf-bebd-4119-9361-1a4fed1af776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:48.694340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-8f33aacf-bebd-4119-9361-1a4fed1af776 None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:48.695591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8f33aacf-bebd-4119-9361-1a4fed1af776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:48.759556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1247/2530] 10.222.0.98 () {58 vars in 967 bytes} [Fri Aug 26 15:56:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:48.810272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a55645-6020-4367-8641-132caa65b5a3 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:48.812605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e0a55645-6020-4367-8641-132caa65b5a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:48.895005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a55645-6020-4367-8641-132caa65b5a3 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:48.900119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d8072-ac6e-4a50-8a51-c0ccffec5a0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:48.900747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d8072-ac6e-4a50-8a51-c0ccffec5a0f service swift] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:48.901029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d8072-ac6e-4a50-8a51-c0ccffec5a0f service swift] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:48.901306 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d8072-ac6e-4a50-8a51-c0ccffec5a0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:48.910199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d8072-ac6e-4a50-8a51-c0ccffec5a0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:48.912416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d8072-ac6e-4a50-8a51-c0ccffec5a0f service swift] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:48.917739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:48.922560 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1284/2531] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:56:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:56:51.828344 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33aacf-bebd-4119-9361-1a4fed1af776 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:51.922050 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33aacf-bebd-4119-9361-1a4fed1af776 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:51.926907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:51.928590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:51.928590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:51.929203 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] PATH_INFO: `/v3/users/bcf7dbeb06404ed39e537d437cb2bb55` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:51.930447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf7dbeb06404ed39e537d437cb2bb55)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:51.932416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:52.103445 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf7dbeb06404ed39e537d437cb2bb55'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:52.104462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf7dbeb06404ed39e537d437cb2bb55'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:52.125361 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf7dbeb06404ed39e537d437cb2bb55'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:52.145255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:52.188864 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf7dbeb06404ed39e537d437cb2bb55'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:52.197609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf7dbeb06404ed39e537d437cb2bb55'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:52.204822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1248/2532] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:56:51 2022] DELETE /identity/v3/users/bcf7dbeb06404ed39e537d437cb2bb55 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:52.209650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e15da8-f731-4e6e-9012-c91fa929e64d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:52.210143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e15da8-f731-4e6e-9012-c91fa929e64d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:52.210244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e15da8-f731-4e6e-9012-c91fa929e64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:52.239315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-a9e15da8-f731-4e6e-9012-c91fa929e64d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:52.241892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a9e15da8-f731-4e6e-9012-c91fa929e64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:52.336994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1285/2533] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:52.342945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:52.344795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:52.443672 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d632fd7-9e02-4285-bb71-6fa34f8681e2 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:52.453385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a004c31-25cb-45d9-abb4-f811e1fc61b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:52.456986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a004c31-25cb-45d9-abb4-f811e1fc61b6 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:52.456986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a004c31-25cb-45d9-abb4-f811e1fc61b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:52.456986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a004c31-25cb-45d9-abb4-f811e1fc61b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:52.467817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a004c31-25cb-45d9-abb4-f811e1fc61b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:52.469436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a004c31-25cb-45d9-abb4-f811e1fc61b6 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:52.483237 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1249/2534] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:52.959666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e15da8-f731-4e6e-9012-c91fa929e64d None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:52.961131 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a9e15da8-f731-4e6e-9012-c91fa929e64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:52.976016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e15da8-f731-4e6e-9012-c91fa929e64d None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:52.979652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:52.980362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:52.980617 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:52.980894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] PATH_INFO: `/v3/projects/1b6d6b93ae5e4ee6b0893b59a2f98136` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:52.982024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b6d6b93ae5e4ee6b0893b59a2f98136)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:52.984024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:53.103610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b6d6b93ae5e4ee6b0893b59a2f98136', 'request_id': 'req-153857dd-a8b9-4a4c-9151-f0e9f423f910'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:53.194508 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b6d6b93ae5e4ee6b0893b59a2f98136', 'request_id': 'req-153857dd-a8b9-4a4c-9151-f0e9f423f910'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:53.201535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b6d6b93ae5e4ee6b0893b59a2f98136', 'request_id': 'req-153857dd-a8b9-4a4c-9151-f0e9f423f910'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:53.214130 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] The token cache is being invalidate because project 1b6d6b93ae5e4ee6b0893b59a2f98136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:56:53.214771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:53.215100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:53.217010 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1286/2535] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:56:52 2022] DELETE /identity/v3/projects/1b6d6b93ae5e4ee6b0893b59a2f98136 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:53.220964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a004c31-25cb-45d9-abb4-f811e1fc61b6 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:53.230962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a004c31-25cb-45d9-abb4-f811e1fc61b6 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:53.294038 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:53.294674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:53.294956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:53.295236 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] PATH_INFO: `/v3/users/95beaa3b8bd04e1eb0558445b8819d39` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:53.296404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=95beaa3b8bd04e1eb0558445b8819d39)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:53.298322 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:53.460742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95beaa3b8bd04e1eb0558445b8819d39'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:53.461472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95beaa3b8bd04e1eb0558445b8819d39'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:53.475114 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95beaa3b8bd04e1eb0558445b8819d39'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:53.542797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95beaa3b8bd04e1eb0558445b8819d39'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:53.554545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95beaa3b8bd04e1eb0558445b8819d39'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:53.565133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1250/2536] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:56:53 2022] DELETE /identity/v3/users/95beaa3b8bd04e1eb0558445b8819d39 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:54.236257 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:54.237847 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:54.305752 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:54.309098 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:54.322931 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:54.347017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153857dd-a8b9-4a4c-9151-f0e9f423f910 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:54.393990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:54.403374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03cc55-a873-456d-876b-3337c3394efe admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:54.409800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:54.410746 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:54.413485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:54.413485 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] PATH_INFO: `/v3/projects/e70bb23e956d4893a3e22092df6a7b03` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:54.414427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70bb23e956d4893a3e22092df6a7b03)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:54.416543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:54.443347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25763d9-7cbb-4095-a1f5-359b6a36be56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:54.444349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25763d9-7cbb-4095-a1f5-359b6a36be56 service placement] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:54.444628 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25763d9-7cbb-4095-a1f5-359b6a36be56 service placement] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:54.444898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25763d9-7cbb-4095-a1f5-359b6a36be56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:54.446903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a25763d9-7cbb-4095-a1f5-359b6a36be56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:54.461369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25763d9-7cbb-4095-a1f5-359b6a36be56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:54.464048 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25763d9-7cbb-4095-a1f5-359b6a36be56 service placement] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:54.475191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1287/2537] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:54.511414 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70bb23e956d4893a3e22092df6a7b03', 'request_id': 'req-27601f39-a88f-4fbd-8309-633ef7db1f99'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:54.517453 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70bb23e956d4893a3e22092df6a7b03', 'request_id': 'req-27601f39-a88f-4fbd-8309-633ef7db1f99'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:54.524635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70bb23e956d4893a3e22092df6a7b03', 'request_id': 'req-27601f39-a88f-4fbd-8309-633ef7db1f99'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:54.588115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] The token cache is being invalidate because project e70bb23e956d4893a3e22092df6a7b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:56:54.588544 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:54.589298 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-27601f39-a88f-4fbd-8309-633ef7db1f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:54.591082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1251/2538] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:56:54 2022] DELETE /identity/v3/projects/e70bb23e956d4893a3e22092df6a7b03 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:56:54.596107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4eaf4-b6e6-4739-88fb-5eeaa388d995 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:54.596397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4eaf4-b6e6-4739-88fb-5eeaa388d995 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:54.596768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4eaf4-b6e6-4739-88fb-5eeaa388d995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:54.622586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-bfa4eaf4-b6e6-4739-88fb-5eeaa388d995 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:54.624181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bfa4eaf4-b6e6-4739-88fb-5eeaa388d995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:54.692342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1288/2539] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:54.700725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa176f7-357f-4097-b2cc-3b2725085e34 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:54.701013 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa176f7-357f-4097-b2cc-3b2725085e34 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:54.701290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa176f7-357f-4097-b2cc-3b2725085e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:54.723062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-5fa176f7-357f-4097-b2cc-3b2725085e34 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:54.724948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5fa176f7-357f-4097-b2cc-3b2725085e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:54.737248 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1252/2540] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:54.738513 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4eaf4-b6e6-4739-88fb-5eeaa388d995 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:54.751735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4eaf4-b6e6-4739-88fb-5eeaa388d995 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:54.765205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239966c3-ba5d-44c3-8174-b227dcbbe848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:54.765205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239966c3-ba5d-44c3-8174-b227dcbbe848 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:54.765205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239966c3-ba5d-44c3-8174-b227dcbbe848 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:54.765205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239966c3-ba5d-44c3-8174-b227dcbbe848 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:54.765843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239966c3-ba5d-44c3-8174-b227dcbbe848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:56:54.765843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239966c3-ba5d-44c3-8174-b227dcbbe848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:54.765843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239966c3-ba5d-44c3-8174-b227dcbbe848 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:54.783910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1289/2541] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:56:54.790534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa176f7-357f-4097-b2cc-3b2725085e34 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:54.799256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa176f7-357f-4097-b2cc-3b2725085e34 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:54.805297 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11857ad2-e8e5-4360-9d14-50a5422763ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:54.805943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11857ad2-e8e5-4360-9d14-50a5422763ea admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:54.806229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11857ad2-e8e5-4360-9d14-50a5422763ea admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:54.806497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11857ad2-e8e5-4360-9d14-50a5422763ea admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:54.809112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11857ad2-e8e5-4360-9d14-50a5422763ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:54.823727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11857ad2-e8e5-4360-9d14-50a5422763ea admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:54.925952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1253/2542] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:54 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:54.926203 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239966c3-ba5d-44c3-8174-b227dcbbe848 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:54.943897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239966c3-ba5d-44c3-8174-b227dcbbe848 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:55.047132 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc6de7-c1ed-4814-bb77-4a3de71372b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:55.048189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc6de7-c1ed-4814-bb77-4a3de71372b1 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:55.048696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc6de7-c1ed-4814-bb77-4a3de71372b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:55.052754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc6de7-c1ed-4814-bb77-4a3de71372b1 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:55.052754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc6de7-c1ed-4814-bb77-4a3de71372b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:55.053897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc6de7-c1ed-4814-bb77-4a3de71372b1 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:55.158646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1290/2543] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:54 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:56:55.163541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11857ad2-e8e5-4360-9d14-50a5422763ea admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:55.174100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11857ad2-e8e5-4360-9d14-50a5422763ea admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:55.179477 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eab85-1b08-4827-9534-0d7e55f7d306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:55.180156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eab85-1b08-4827-9534-0d7e55f7d306 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:55.180445 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eab85-1b08-4827-9534-0d7e55f7d306 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:55.180728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eab85-1b08-4827-9534-0d7e55f7d306 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:55.181574 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6eab85-1b08-4827-9534-0d7e55f7d306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:55.183056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6eab85-1b08-4827-9534-0d7e55f7d306 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:55.202332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1254/2544] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:55.206584 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc6de7-c1ed-4814-bb77-4a3de71372b1 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:55.219238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc6de7-c1ed-4814-bb77-4a3de71372b1 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:55.226823 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e59f9-52d7-40bd-91d1-c0677616ff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:55.227703 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e59f9-52d7-40bd-91d1-c0677616ff9c admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:55.228166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e59f9-52d7-40bd-91d1-c0677616ff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:55.228586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e59f9-52d7-40bd-91d1-c0677616ff9c admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:55.229579 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e59f9-52d7-40bd-91d1-c0677616ff9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:55.232202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e59f9-52d7-40bd-91d1-c0677616ff9c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:55.249750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1291/2545] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:55.255133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eab85-1b08-4827-9534-0d7e55f7d306 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:55.268022 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eab85-1b08-4827-9534-0d7e55f7d306 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:55.276578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe25884-2bd7-468b-b68b-9ae9a4b328f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:55.277281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe25884-2bd7-468b-b68b-9ae9a4b328f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:55.277574 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe25884-2bd7-468b-b68b-9ae9a4b328f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:55.277867 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe25884-2bd7-468b-b68b-9ae9a4b328f6 admin admin] PATH_INFO: `/v3/projects/1f8f98eb014f40d4a650e70663f76679/users/af155d7975e34c8ba8254b7fd37df08a/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:55.297846 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe25884-2bd7-468b-b68b-9ae9a4b328f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f8f98eb014f40d4a650e70663f76679, user_id=af155d7975e34c8ba8254b7fd37df08a, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:55.300492 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe25884-2bd7-468b-b68b-9ae9a4b328f6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:55.410299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1255/2546] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:55 2022] PUT /identity/v3/projects/1f8f98eb014f40d4a650e70663f76679/users/af155d7975e34c8ba8254b7fd37df08a/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:55.412750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ae44d-b357-45c8-95e5-05deaeeb9528 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:55.413038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ae44d-b357-45c8-95e5-05deaeeb9528 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:55.413371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ae44d-b357-45c8-95e5-05deaeeb9528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:55.443323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3d2ae44d-b357-45c8-95e5-05deaeeb9528 None None] MFA Rules not processed for user `af155d7975e34c8ba8254b7fd37df08a`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:55.444613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3d2ae44d-b357-45c8-95e5-05deaeeb9528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:55.454987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:55.515772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1292/2547] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:55.520599 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe25884-2bd7-468b-b68b-9ae9a4b328f6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:55.580332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:55.634372 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe25884-2bd7-468b-b68b-9ae9a4b328f6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:55.642549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d71328-e9fd-41df-bed5-3fc03f1ba836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:55.643595 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d71328-e9fd-41df-bed5-3fc03f1ba836 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:55.644131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d71328-e9fd-41df-bed5-3fc03f1ba836 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:55.644649 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d71328-e9fd-41df-bed5-3fc03f1ba836 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:55.645527 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d71328-e9fd-41df-bed5-3fc03f1ba836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:55.647785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d71328-e9fd-41df-bed5-3fc03f1ba836 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:55.792921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1256/2548] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:55 2022] POST /identity/v3/projects => generated 385 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:55.799940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ae44d-b357-45c8-95e5-05deaeeb9528 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:55.813872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ae44d-b357-45c8-95e5-05deaeeb9528 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:55.891475 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fce73-b293-459a-9fe6-aaad0b9e1c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:55.893171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fce73-b293-459a-9fe6-aaad0b9e1c93 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:55.893171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fce73-b293-459a-9fe6-aaad0b9e1c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:55.893171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fce73-b293-459a-9fe6-aaad0b9e1c93 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:55.893657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fce73-b293-459a-9fe6-aaad0b9e1c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:55.895368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fce73-b293-459a-9fe6-aaad0b9e1c93 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:56.260534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1293/2549] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:55 2022] POST /identity/v3/users => generated 299 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:56.263220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d71328-e9fd-41df-bed5-3fc03f1ba836 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:56.274978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d71328-e9fd-41df-bed5-3fc03f1ba836 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:56.280060 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1e1cf-4982-4bf4-b60e-183fc1f24f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:56.280857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1e1cf-4982-4bf4-b60e-183fc1f24f53 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:56.281250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1e1cf-4982-4bf4-b60e-183fc1f24f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:56.281652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1e1cf-4982-4bf4-b60e-183fc1f24f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:56.282435 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1e1cf-4982-4bf4-b60e-183fc1f24f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:56.284746 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1e1cf-4982-4bf4-b60e-183fc1f24f53 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:56.300550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1257/2550] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:56:56.303663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fce73-b293-459a-9fe6-aaad0b9e1c93 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:56.314792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fce73-b293-459a-9fe6-aaad0b9e1c93 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:56.325456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13caa6-4325-450f-b5a3-46549edbdfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:56.326142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13caa6-4325-450f-b5a3-46549edbdfd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:56.326425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13caa6-4325-450f-b5a3-46549edbdfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:56.326700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13caa6-4325-450f-b5a3-46549edbdfd4 admin admin] PATH_INFO: `/v3/projects/b6c20dee88064fbb94978616aeba2506/users/af854e250fdb4f878cc17234a5a9be54/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:56.344511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13caa6-4325-450f-b5a3-46549edbdfd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6c20dee88064fbb94978616aeba2506, user_id=af854e250fdb4f878cc17234a5a9be54, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:56.350149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13caa6-4325-450f-b5a3-46549edbdfd4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:56.591603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1294/2551] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:56 2022] PUT /identity/v3/projects/b6c20dee88064fbb94978616aeba2506/users/af854e250fdb4f878cc17234a5a9be54/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:56.595329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1e1cf-4982-4bf4-b60e-183fc1f24f53 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:56.605989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1e1cf-4982-4bf4-b60e-183fc1f24f53 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:56.700250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8f845-be70-4cd8-9b63-030cf495f7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:56.701265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8f845-be70-4cd8-9b63-030cf495f7e0 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:56.701727 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8f845-be70-4cd8-9b63-030cf495f7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:56.708327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8f845-be70-4cd8-9b63-030cf495f7e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:56.708327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8f845-be70-4cd8-9b63-030cf495f7e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:56.708327 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8f845-be70-4cd8-9b63-030cf495f7e0 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:56.717825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1258/2552] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:56.723931 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13caa6-4325-450f-b5a3-46549edbdfd4 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:56.736791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13caa6-4325-450f-b5a3-46549edbdfd4 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:56.742578 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a38c2-2e41-4a4d-b0c0-1ab69e373598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:56.743249 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a38c2-2e41-4a4d-b0c0-1ab69e373598 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:56.743530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a38c2-2e41-4a4d-b0c0-1ab69e373598 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:56.743808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a38c2-2e41-4a4d-b0c0-1ab69e373598 admin admin] PATH_INFO: `/v3/projects/b6c20dee88064fbb94978616aeba2506/users/af854e250fdb4f878cc17234a5a9be54/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:56.745790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a38c2-2e41-4a4d-b0c0-1ab69e373598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6c20dee88064fbb94978616aeba2506, user_id=af854e250fdb4f878cc17234a5a9be54, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:56.749118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a38c2-2e41-4a4d-b0c0-1ab69e373598 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:56.767362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1295/2553] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:56 2022] PUT /identity/v3/projects/b6c20dee88064fbb94978616aeba2506/users/af854e250fdb4f878cc17234a5a9be54/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:56.779216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e440cc-043a-40de-88c9-b22c427f1bba None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:56.781172 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e440cc-043a-40de-88c9-b22c427f1bba None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:56.781799 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e440cc-043a-40de-88c9-b22c427f1bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:56.834547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-01e440cc-043a-40de-88c9-b22c427f1bba None None] MFA Rules not processed for user `af854e250fdb4f878cc17234a5a9be54`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:56.836869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-01e440cc-043a-40de-88c9-b22c427f1bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:56.937990 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1259/2554] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:56.951782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a38c2-2e41-4a4d-b0c0-1ab69e373598 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:56.954175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c98a38c2-2e41-4a4d-b0c0-1ab69e373598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:57.046298 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a38c2-2e41-4a4d-b0c0-1ab69e373598 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:57.050573 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bf9fc-b22e-44d6-9646-a8161c5848fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:57.051241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bf9fc-b22e-44d6-9646-a8161c5848fc service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:57.051530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bf9fc-b22e-44d6-9646-a8161c5848fc service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:57.051982 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bf9fc-b22e-44d6-9646-a8161c5848fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:57.057487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e440cc-043a-40de-88c9-b22c427f1bba None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:57.059024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-01e440cc-043a-40de-88c9-b22c427f1bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:57.063335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bf9fc-b22e-44d6-9646-a8161c5848fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:57.064868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bf9fc-b22e-44d6-9646-a8161c5848fc service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:57.076469 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1296/2555] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:57.129207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:57.158921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bf9fc-b22e-44d6-9646-a8161c5848fc service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:57.172093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:57.222928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e440cc-043a-40de-88c9-b22c427f1bba None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:57.232824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a72165-3dfa-4470-bfab-188acc651af8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:57.233375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72165-3dfa-4470-bfab-188acc651af8 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:57.233664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72165-3dfa-4470-bfab-188acc651af8 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:57.233942 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72165-3dfa-4470-bfab-188acc651af8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:57.235883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-87a72165-3dfa-4470-bfab-188acc651af8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:57.247333 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bf9fc-b22e-44d6-9646-a8161c5848fc service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:57.252827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:57.253538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:57.253894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:57.254278 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] PATH_INFO: `/v3/users/af155d7975e34c8ba8254b7fd37df08a` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:57.255542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af155d7975e34c8ba8254b7fd37df08a)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:57.257697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:57.303665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a72165-3dfa-4470-bfab-188acc651af8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:57.305464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a72165-3dfa-4470-bfab-188acc651af8 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:57.317432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1260/2556] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:56:57.468335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af155d7975e34c8ba8254b7fd37df08a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:57.484683 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af155d7975e34c8ba8254b7fd37df08a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:57.485995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af155d7975e34c8ba8254b7fd37df08a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:57.486743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af155d7975e34c8ba8254b7fd37df08a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:57.584254 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af155d7975e34c8ba8254b7fd37df08a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:57.609510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9e8a3-e398-42a6-932e-88c12c8e80a9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:57.610031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9e8a3-e398-42a6-932e-88c12c8e80a9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:57.610119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9e8a3-e398-42a6-932e-88c12c8e80a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:57.612751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1297/2557] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:56:57 2022] DELETE /identity/v3/users/af155d7975e34c8ba8254b7fd37df08a => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:57.631921 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-c5b9e8a3-e398-42a6-932e-88c12c8e80a9 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:57.633329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c5b9e8a3-e398-42a6-932e-88c12c8e80a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:57.717854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1261/2558] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:57.729118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:57.730570 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:57.790609 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8d69b-4fc7-4ae0-8059-d91195ed46f6 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:57.794382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb280c-5186-4f8a-b78a-028d877805b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:57.794995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb280c-5186-4f8a-b78a-028d877805b8 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:57.795280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb280c-5186-4f8a-b78a-028d877805b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:57.795557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb280c-5186-4f8a-b78a-028d877805b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:57.805866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb280c-5186-4f8a-b78a-028d877805b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:57.807297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb280c-5186-4f8a-b78a-028d877805b8 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:57.816962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1298/2559] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:56:58.439960 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9e8a3-e398-42a6-932e-88c12c8e80a9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:58.442023 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c5b9e8a3-e398-42a6-932e-88c12c8e80a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:58.474331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9e8a3-e398-42a6-932e-88c12c8e80a9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:58.488881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:58.488881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:58.488881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:58.488881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] PATH_INFO: `/v3/projects/1f8f98eb014f40d4a650e70663f76679` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:58.492535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f8f98eb014f40d4a650e70663f76679)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:58.492535 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:58.649985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f8f98eb014f40d4a650e70663f76679', 'request_id': 'req-90a21cdf-4163-4d30-b305-bd5a9f7545d1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:58.656393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f8f98eb014f40d4a650e70663f76679', 'request_id': 'req-90a21cdf-4163-4d30-b305-bd5a9f7545d1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:58.664597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f8f98eb014f40d4a650e70663f76679', 'request_id': 'req-90a21cdf-4163-4d30-b305-bd5a9f7545d1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:58.666723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:58.808662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] The token cache is being invalidate because project 1f8f98eb014f40d4a650e70663f76679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:56:58.808961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:58.809838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:58.815403 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1262/2560] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:56:58 2022] DELETE /identity/v3/projects/1f8f98eb014f40d4a650e70663f76679 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:58.816279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb280c-5186-4f8a-b78a-028d877805b8 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:58.828889 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb280c-5186-4f8a-b78a-028d877805b8 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:58.862328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:58.896099 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:58.897404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:58.897695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:58.897966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] PATH_INFO: `/v3/users/af854e250fdb4f878cc17234a5a9be54` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:58.899269 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af854e250fdb4f878cc17234a5a9be54)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:58.902133 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:59.132847 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af854e250fdb4f878cc17234a5a9be54'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.146343 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af854e250fdb4f878cc17234a5a9be54'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.148502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af854e250fdb4f878cc17234a5a9be54'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.149785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af854e250fdb4f878cc17234a5a9be54'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.243705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af854e250fdb4f878cc17234a5a9be54'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.260518 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1299/2561] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:56:58 2022] DELETE /identity/v3/users/af854e250fdb4f878cc17234a5a9be54 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:56:59.645463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:59.646873 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:59.666968 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a21cdf-4163-4d30-b305-bd5a9f7545d1 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:59.780581 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d0e9b-96e5-4e14-abfc-f64b9c258751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:59.781342 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0e9b-96e5-4e14-abfc-f64b9c258751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:59.781526 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0e9b-96e5-4e14-abfc-f64b9c258751 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:59.781785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0e9b-96e5-4e14-abfc-f64b9c258751 admin admin] PATH_INFO: `/v3/users/b0eb3bddc1bd4557a280ad21f9092093` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:59.783234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d0e9b-96e5-4e14-abfc-f64b9c258751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0eb3bddc1bd4557a280ad21f9092093)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:59.785397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d0e9b-96e5-4e14-abfc-f64b9c258751 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:59.941563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:59.943173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:59.956688 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab65fc-cf23-40db-a32b-817e9dd9fab6 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:59.962929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:59.962929 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:59.964092 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:59.964092 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] PATH_INFO: `/v3/projects/b6c20dee88064fbb94978616aeba2506` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:59.965206 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6c20dee88064fbb94978616aeba2506)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:59.967155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:00.076437 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-507d0e9b-96e5-4e14-abfc-f64b9c258751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0eb3bddc1bd4557a280ad21f9092093'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.077091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-507d0e9b-96e5-4e14-abfc-f64b9c258751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0eb3bddc1bd4557a280ad21f9092093'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.090003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-507d0e9b-96e5-4e14-abfc-f64b9c258751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0eb3bddc1bd4557a280ad21f9092093'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.134251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6c20dee88064fbb94978616aeba2506', 'request_id': 'req-b0545f0e-be50-4701-a89f-66293dd5bb4b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.177332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-507d0e9b-96e5-4e14-abfc-f64b9c258751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0eb3bddc1bd4557a280ad21f9092093'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.185913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-507d0e9b-96e5-4e14-abfc-f64b9c258751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0eb3bddc1bd4557a280ad21f9092093'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.192647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1263/2562] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:56:59 2022] DELETE /identity/v3/users/b0eb3bddc1bd4557a280ad21f9092093 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:00.263455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6c20dee88064fbb94978616aeba2506', 'request_id': 'req-b0545f0e-be50-4701-a89f-66293dd5bb4b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.277566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6c20dee88064fbb94978616aeba2506', 'request_id': 'req-b0545f0e-be50-4701-a89f-66293dd5bb4b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.293371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] The token cache is being invalidate because project b6c20dee88064fbb94978616aeba2506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:00.293874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.295948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-b0545f0e-be50-4701-a89f-66293dd5bb4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.296209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1300/2563] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:56:59 2022] DELETE /identity/v3/projects/b6c20dee88064fbb94978616aeba2506 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:00.301962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb9ed7-c236-48e6-a022-8354b2d70504 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.302258 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb9ed7-c236-48e6-a022-8354b2d70504 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.302573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb9ed7-c236-48e6-a022-8354b2d70504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.323972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-f3bb9ed7-c236-48e6-a022-8354b2d70504 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:00.325313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f3bb9ed7-c236-48e6-a022-8354b2d70504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:00.395341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1264/2564] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:00.404863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43b2e6-28cd-42e1-a3f5-54a6f48a5ba4 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.405310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43b2e6-28cd-42e1-a3f5-54a6f48a5ba4 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.405420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43b2e6-28cd-42e1-a3f5-54a6f48a5ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.425841 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9d43b2e6-28cd-42e1-a3f5-54a6f48a5ba4 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:00.427100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9d43b2e6-28cd-42e1-a3f5-54a6f48a5ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:00.435358 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1301/2565] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:00.439929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb9ed7-c236-48e6-a022-8354b2d70504 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:00.449896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb9ed7-c236-48e6-a022-8354b2d70504 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:00.454911 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55e1c0-49a1-468a-a497-18e56480233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:00.455517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55e1c0-49a1-468a-a497-18e56480233e admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.455795 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55e1c0-49a1-468a-a497-18e56480233e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.456115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55e1c0-49a1-468a-a497-18e56480233e admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.456763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55e1c0-49a1-468a-a497-18e56480233e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:00.466240 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55e1c0-49a1-468a-a497-18e56480233e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:00.466240 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55e1c0-49a1-468a-a497-18e56480233e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:00.466240 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:00.473400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1265/2566] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:00.477117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43b2e6-28cd-42e1-a3f5-54a6f48a5ba4 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:00.486898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43b2e6-28cd-42e1-a3f5-54a6f48a5ba4 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:00.500713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd25e5c-976d-4677-b333-706194c322e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:00.500713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd25e5c-976d-4677-b333-706194c322e6 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.500713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd25e5c-976d-4677-b333-706194c322e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.502783 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd25e5c-976d-4677-b333-706194c322e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.503431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd25e5c-976d-4677-b333-706194c322e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:00.504092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55e1c0-49a1-468a-a497-18e56480233e admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:00.505964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ac55e1c0-49a1-468a-a497-18e56480233e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:00.506247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd25e5c-976d-4677-b333-706194c322e6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:00.527577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55e1c0-49a1-468a-a497-18e56480233e admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:00.532578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:00.533371 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.533845 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.534285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] PATH_INFO: `/v3/projects/1b64a7d4a15a4d019f7bfc17a59889a5` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.535473 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b64a7d4a15a4d019f7bfc17a59889a5)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:00.538341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:00.679363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1302/2567] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:00 2022] POST /identity/v3/projects => generated 387 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:00.686895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd25e5c-976d-4677-b333-706194c322e6 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:00.689452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b64a7d4a15a4d019f7bfc17a59889a5', 'request_id': 'req-e2c68b18-b091-4281-b7f4-bb8210019871'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.692644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:00.697823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b64a7d4a15a4d019f7bfc17a59889a5', 'request_id': 'req-e2c68b18-b091-4281-b7f4-bb8210019871'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.705200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b64a7d4a15a4d019f7bfc17a59889a5', 'request_id': 'req-e2c68b18-b091-4281-b7f4-bb8210019871'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.757251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd25e5c-976d-4677-b333-706194c322e6 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:00.764396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a1a05-ae07-425d-9a5d-8b79c5935eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:00.765036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a1a05-ae07-425d-9a5d-8b79c5935eca admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.765325 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a1a05-ae07-425d-9a5d-8b79c5935eca admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.765605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a1a05-ae07-425d-9a5d-8b79c5935eca admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.766240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a1a05-ae07-425d-9a5d-8b79c5935eca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:00.768213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a1a05-ae07-425d-9a5d-8b79c5935eca admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:00.811566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] The token cache is being invalidate because project 1b64a7d4a15a4d019f7bfc17a59889a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:00.811943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.812609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.814320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1266/2568] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:00 2022] DELETE /identity/v3/projects/1b64a7d4a15a4d019f7bfc17a59889a5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:00.817989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:00.819838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:00.899006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c68b18-b091-4281-b7f4-bb8210019871 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:00.903841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:00.904510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.904801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.905077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] PATH_INFO: `/v3/users/496f6511485d476baddb2ea38531e772` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.906220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=496f6511485d476baddb2ea38531e772)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:00.908255 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:00.915814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1303/2569] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:00 2022] POST /identity/v3/users => generated 294 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:00.920224 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a1a05-ae07-425d-9a5d-8b79c5935eca admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:00.924368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-620a1a05-ae07-425d-9a5d-8b79c5935eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:00.937436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a1a05-ae07-425d-9a5d-8b79c5935eca admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:00.942292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d9c0e-4239-4641-9076-46f67cb65f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:00.942890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d9c0e-4239-4641-9076-46f67cb65f99 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.943676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d9c0e-4239-4641-9076-46f67cb65f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.943996 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d9c0e-4239-4641-9076-46f67cb65f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.944646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d9c0e-4239-4641-9076-46f67cb65f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:00.946314 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d9c0e-4239-4641-9076-46f67cb65f99 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:00.956818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1304/2570] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:00.961538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d9c0e-4239-4641-9076-46f67cb65f99 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:00.971170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d9c0e-4239-4641-9076-46f67cb65f99 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:00.975978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33889301-1b15-4f8d-a8ad-0253c6256486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:00.987941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33889301-1b15-4f8d-a8ad-0253c6256486 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.987941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33889301-1b15-4f8d-a8ad-0253c6256486 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.987941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33889301-1b15-4f8d-a8ad-0253c6256486 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.987941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33889301-1b15-4f8d-a8ad-0253c6256486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:00.987941 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33889301-1b15-4f8d-a8ad-0253c6256486 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:00.990638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1305/2571] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:00.994803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33889301-1b15-4f8d-a8ad-0253c6256486 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.004902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33889301-1b15-4f8d-a8ad-0253c6256486 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:01.010223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fa539-8c75-45b5-8b97-d1160e275be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:01.010817 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fa539-8c75-45b5-8b97-d1160e275be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.011089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fa539-8c75-45b5-8b97-d1160e275be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.011363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fa539-8c75-45b5-8b97-d1160e275be5 admin admin] PATH_INFO: `/v3/projects/d5042734dfaf47d2b80bdd20c9daf3f0/users/a09bd553f0974376b550e0a7b801b083/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.026472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fa539-8c75-45b5-8b97-d1160e275be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5042734dfaf47d2b80bdd20c9daf3f0, user_id=a09bd553f0974376b550e0a7b801b083, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:01.029302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fa539-8c75-45b5-8b97-d1160e275be5 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:01.120338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1306/2572] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:00 2022] PUT /identity/v3/projects/d5042734dfaf47d2b80bdd20c9daf3f0/users/a09bd553f0974376b550e0a7b801b083/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:01.135342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b740a-2213-47ff-89de-b84874693224 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.135627 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b740a-2213-47ff-89de-b84874693224 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.135955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b740a-2213-47ff-89de-b84874693224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.163294 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-121b740a-2213-47ff-89de-b84874693224 None None] MFA Rules not processed for user `a09bd553f0974376b550e0a7b801b083`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:01.164546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-121b740a-2213-47ff-89de-b84874693224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:01.189572 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '496f6511485d476baddb2ea38531e772'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:01.190256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '496f6511485d476baddb2ea38531e772'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:01.213550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '496f6511485d476baddb2ea38531e772'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:01.233058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1307/2573] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:01.237892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b740a-2213-47ff-89de-b84874693224 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.306205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b740a-2213-47ff-89de-b84874693224 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:01.312953 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330afdd-f6f1-4b74-96fd-2f5ee749edf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:01.313859 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330afdd-f6f1-4b74-96fd-2f5ee749edf2 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.314291 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330afdd-f6f1-4b74-96fd-2f5ee749edf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.315022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330afdd-f6f1-4b74-96fd-2f5ee749edf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.316268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330afdd-f6f1-4b74-96fd-2f5ee749edf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:01.320625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330afdd-f6f1-4b74-96fd-2f5ee749edf2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:01.324937 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '496f6511485d476baddb2ea38531e772'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:01.336226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '496f6511485d476baddb2ea38531e772'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:01.348455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1267/2574] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:00 2022] DELETE /identity/v3/users/496f6511485d476baddb2ea38531e772 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:01.431542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1308/2575] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:01 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:01.438655 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.440329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:01.518112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a4fde-3596-4eb8-a2e2-053a14486ef6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:01.596057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cc869-a2a9-446a-9e4a-9e3653bd00cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:01.597036 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cc869-a2a9-446a-9e4a-9e3653bd00cc admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.597651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cc869-a2a9-446a-9e4a-9e3653bd00cc admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.598901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cc869-a2a9-446a-9e4a-9e3653bd00cc admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.600673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cc869-a2a9-446a-9e4a-9e3653bd00cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:01.604016 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cc869-a2a9-446a-9e4a-9e3653bd00cc admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:01.663401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330afdd-f6f1-4b74-96fd-2f5ee749edf2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.665678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f330afdd-f6f1-4b74-96fd-2f5ee749edf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:01.680064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330afdd-f6f1-4b74-96fd-2f5ee749edf2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:01.684876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:01.685521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.685799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.686070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] PATH_INFO: `/v3/projects/f357910be96c43b1bbdea77c6f9d9152` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.687242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f357910be96c43b1bbdea77c6f9d9152)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:01.689396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:01.760340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1268/2576] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:01 2022] POST /identity/v3/users => generated 293 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:01.764402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cc869-a2a9-446a-9e4a-9e3653bd00cc admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.775253 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cc869-a2a9-446a-9e4a-9e3653bd00cc admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:01.780042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ed007-9460-4d80-9501-d825cf31be48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:01.780650 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed007-9460-4d80-9501-d825cf31be48 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.780932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed007-9460-4d80-9501-d825cf31be48 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.781212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed007-9460-4d80-9501-d825cf31be48 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.781860 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ed007-9460-4d80-9501-d825cf31be48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:01.783461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ed007-9460-4d80-9501-d825cf31be48 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:01.794440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1269/2577] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:01.798767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ed007-9460-4d80-9501-d825cf31be48 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.810510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ed007-9460-4d80-9501-d825cf31be48 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:01.816495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7043738-749a-4aad-b59d-2575179d0538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:01.817112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7043738-749a-4aad-b59d-2575179d0538 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.817464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7043738-749a-4aad-b59d-2575179d0538 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.817879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7043738-749a-4aad-b59d-2575179d0538 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.818570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7043738-749a-4aad-b59d-2575179d0538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:01.820281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7043738-749a-4aad-b59d-2575179d0538 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:01.835487 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1270/2578] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:01.836680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7043738-749a-4aad-b59d-2575179d0538 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.839810 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f357910be96c43b1bbdea77c6f9d9152', 'request_id': 'req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:01.849015 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7043738-749a-4aad-b59d-2575179d0538 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:01.911680 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba972d-6b1d-4cb6-84a7-b724e286e6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:01.912947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba972d-6b1d-4cb6-84a7-b724e286e6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.913230 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba972d-6b1d-4cb6-84a7-b724e286e6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.913505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba972d-6b1d-4cb6-84a7-b724e286e6e4 admin admin] PATH_INFO: `/v3/projects/9a0905ad363d4221896e7e8e904dfa4d/users/08454683053f4c00ad4b26505c706949/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.918683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:01.930638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba972d-6b1d-4cb6-84a7-b724e286e6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0905ad363d4221896e7e8e904dfa4d, user_id=08454683053f4c00ad4b26505c706949, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:01.936065 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba972d-6b1d-4cb6-84a7-b724e286e6e4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:01.957974 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f357910be96c43b1bbdea77c6f9d9152', 'request_id': 'req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:01.965743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f357910be96c43b1bbdea77c6f9d9152', 'request_id': 'req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:01.977030 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] The token cache is being invalidate because project f357910be96c43b1bbdea77c6f9d9152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:01.977330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:01.977997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-761ac3f8-fe6e-4cdf-a973-ae087e5ad553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:01.979502 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1309/2579] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:01 2022] DELETE /identity/v3/projects/f357910be96c43b1bbdea77c6f9d9152 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:02.036391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1271/2580] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:01 2022] PUT /identity/v3/projects/9a0905ad363d4221896e7e8e904dfa4d/users/08454683053f4c00ad4b26505c706949/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:02.046556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:02.048808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5eedcc-425f-404e-bf37-ac5cf1ce552f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.049367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5eedcc-425f-404e-bf37-ac5cf1ce552f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.049743 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5eedcc-425f-404e-bf37-ac5cf1ce552f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.086552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-6b5eedcc-425f-404e-bf37-ac5cf1ce552f None None] MFA Rules not processed for user `08454683053f4c00ad4b26505c706949`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:02.087983 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6b5eedcc-425f-404e-bf37-ac5cf1ce552f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:02.175202 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1310/2581] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:02.179894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba972d-6b1d-4cb6-84a7-b724e286e6e4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:02.182162 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-07ba972d-6b1d-4cb6-84a7-b724e286e6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:02.265456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba972d-6b1d-4cb6-84a7-b724e286e6e4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:02.271229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1896bb-1a01-440f-aaec-59ae74d111c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:02.272943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1896bb-1a01-440f-aaec-59ae74d111c8 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.273103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1896bb-1a01-440f-aaec-59ae74d111c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.273334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1896bb-1a01-440f-aaec-59ae74d111c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.274835 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1896bb-1a01-440f-aaec-59ae74d111c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:02.276796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1896bb-1a01-440f-aaec-59ae74d111c8 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:02.377194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1272/2582] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:02 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:02.382661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5eedcc-425f-404e-bf37-ac5cf1ce552f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:02.394524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5eedcc-425f-404e-bf37-ac5cf1ce552f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:02.456769 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65942b5a-4b3d-4ede-8b45-c6714d55c690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:02.457400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65942b5a-4b3d-4ede-8b45-c6714d55c690 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.457684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65942b5a-4b3d-4ede-8b45-c6714d55c690 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.457964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65942b5a-4b3d-4ede-8b45-c6714d55c690 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.458606 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65942b5a-4b3d-4ede-8b45-c6714d55c690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:02.461174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65942b5a-4b3d-4ede-8b45-c6714d55c690 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:02.518445 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:02.589292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1311/2583] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:02 2022] POST /identity/v3/users => generated 300 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:02.593308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1896bb-1a01-440f-aaec-59ae74d111c8 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:02.603833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1896bb-1a01-440f-aaec-59ae74d111c8 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:02.609092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc628514-1b8d-412f-83d5-84249870f849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:02.609794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc628514-1b8d-412f-83d5-84249870f849 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.610109 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc628514-1b8d-412f-83d5-84249870f849 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.610496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc628514-1b8d-412f-83d5-84249870f849 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.611158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc628514-1b8d-412f-83d5-84249870f849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:02.613025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc628514-1b8d-412f-83d5-84249870f849 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:02.627924 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1273/2584] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:02.629165 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65942b5a-4b3d-4ede-8b45-c6714d55c690 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:02.640340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65942b5a-4b3d-4ede-8b45-c6714d55c690 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:02.645242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac5f1d-0983-4ad5-a093-5aceb7a5b262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:02.645987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac5f1d-0983-4ad5-a093-5aceb7a5b262 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.646379 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac5f1d-0983-4ad5-a093-5aceb7a5b262 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.646657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac5f1d-0983-4ad5-a093-5aceb7a5b262 admin admin] PATH_INFO: `/v3/projects/087001e6693e425d8d072ef6dcb78de0/users/61b92c8453a342b8891ab30355ca2b8d/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.663281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac5f1d-0983-4ad5-a093-5aceb7a5b262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087001e6693e425d8d072ef6dcb78de0, user_id=61b92c8453a342b8891ab30355ca2b8d, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:02.668321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac5f1d-0983-4ad5-a093-5aceb7a5b262 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:02.747851 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1312/2585] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:02 2022] PUT /identity/v3/projects/087001e6693e425d8d072ef6dcb78de0/users/61b92c8453a342b8891ab30355ca2b8d/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:02.752143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc628514-1b8d-412f-83d5-84249870f849 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:02.763345 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc628514-1b8d-412f-83d5-84249870f849 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:02.793287 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d167f6-195e-4271-acca-3720b022e7a8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.793571 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d167f6-195e-4271-acca-3720b022e7a8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.793854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d167f6-195e-4271-acca-3720b022e7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.814601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-15d167f6-195e-4271-acca-3720b022e7a8 None None] MFA Rules not processed for user `9348310fd06b473786d1c6be4f20c481`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:02.815890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-15d167f6-195e-4271-acca-3720b022e7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:02.822883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cfdfa-d86f-44da-b7d9-2d01111701be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:02.823504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cfdfa-d86f-44da-b7d9-2d01111701be admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.823824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cfdfa-d86f-44da-b7d9-2d01111701be admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.824129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cfdfa-d86f-44da-b7d9-2d01111701be admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.824834 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20cfdfa-d86f-44da-b7d9-2d01111701be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:02.826848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20cfdfa-d86f-44da-b7d9-2d01111701be admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:02.838239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1274/2586] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:02.847419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cfdfa-d86f-44da-b7d9-2d01111701be admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:02.857575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cfdfa-d86f-44da-b7d9-2d01111701be admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:02.864441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198231c-6ff1-47c2-bd3f-b86e66f87f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:02.866493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198231c-6ff1-47c2-bd3f-b86e66f87f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.866493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198231c-6ff1-47c2-bd3f-b86e66f87f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.866493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198231c-6ff1-47c2-bd3f-b86e66f87f3f admin admin] PATH_INFO: `/v3/projects/087001e6693e425d8d072ef6dcb78de0/users/61b92c8453a342b8891ab30355ca2b8d/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.867820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198231c-6ff1-47c2-bd3f-b86e66f87f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=087001e6693e425d8d072ef6dcb78de0, user_id=61b92c8453a342b8891ab30355ca2b8d, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:02.871153 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198231c-6ff1-47c2-bd3f-b86e66f87f3f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:02.880095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:02.885376 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1275/2587] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:02 2022] PUT /identity/v3/projects/087001e6693e425d8d072ef6dcb78de0/users/61b92c8453a342b8891ab30355ca2b8d/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:02.888803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1313/2588] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:02.902589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2afd9-417d-40de-864f-3340ca1d2168 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.902589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2afd9-417d-40de-864f-3340ca1d2168 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.902849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2afd9-417d-40de-864f-3340ca1d2168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.935798 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-4de2afd9-417d-40de-864f-3340ca1d2168 None None] MFA Rules not processed for user `61b92c8453a342b8891ab30355ca2b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:02.937143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4de2afd9-417d-40de-864f-3340ca1d2168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:03.014088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1276/2589] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:03.025267 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d167f6-195e-4271-acca-3720b022e7a8 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:03.026762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-15d167f6-195e-4271-acca-3720b022e7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:03.088805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d167f6-195e-4271-acca-3720b022e7a8 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:03.141546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead15457-559a-4b9a-ace9-7b28733f6ad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:03.142169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead15457-559a-4b9a-ace9-7b28733f6ad6 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:03.142452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead15457-559a-4b9a-ace9-7b28733f6ad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:03.142727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead15457-559a-4b9a-ace9-7b28733f6ad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:03.154740 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead15457-559a-4b9a-ace9-7b28733f6ad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:03.156276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead15457-559a-4b9a-ace9-7b28733f6ad6 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:03.166415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1314/2590] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:03.227131 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:04.516967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2afd9-417d-40de-864f-3340ca1d2168 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:04.526956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2afd9-417d-40de-864f-3340ca1d2168 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:04.530593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5dd90-4a77-4980-8003-9e137c780bac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:04.531210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5dd90-4a77-4980-8003-9e137c780bac service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:04.531492 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5dd90-4a77-4980-8003-9e137c780bac service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:04.531765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5dd90-4a77-4980-8003-9e137c780bac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:04.533595 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7ad5dd90-4a77-4980-8003-9e137c780bac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:04.598984 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5dd90-4a77-4980-8003-9e137c780bac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:04.600967 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5dd90-4a77-4980-8003-9e137c780bac service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:04.611095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1277/2591] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:10.219657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead15457-559a-4b9a-ace9-7b28733f6ad6 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:10.221133 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ead15457-559a-4b9a-ace9-7b28733f6ad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:10.228218 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:10.233260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead15457-559a-4b9a-ace9-7b28733f6ad6 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:10.236906 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499f664-3c47-47e6-a2c4-cb0455a7a252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:10.237489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499f664-3c47-47e6-a2c4-cb0455a7a252 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:10.237770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499f664-3c47-47e6-a2c4-cb0455a7a252 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:10.238072 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499f664-3c47-47e6-a2c4-cb0455a7a252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:10.294565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499f664-3c47-47e6-a2c4-cb0455a7a252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:10.296039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499f664-3c47-47e6-a2c4-cb0455a7a252 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:10.304830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1315/2592] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:14.606003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=37 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:15.256266 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5dd90-4a77-4980-8003-9e137c780bac service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:15.324781 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5dd90-4a77-4980-8003-9e137c780bac service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:15.329902 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:15.330539 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:15.330823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:15.331128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] PATH_INFO: `/v3/users/a09bd553f0974376b550e0a7b801b083` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:15.332321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a09bd553f0974376b550e0a7b801b083)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:15.334279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:15.528332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a09bd553f0974376b550e0a7b801b083'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:15.528998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a09bd553f0974376b550e0a7b801b083'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:15.541101 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a09bd553f0974376b550e0a7b801b083'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:15.621509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a09bd553f0974376b550e0a7b801b083'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:15.629267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a09bd553f0974376b550e0a7b801b083'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:15.636370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1278/2593] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:15 2022] DELETE /identity/v3/users/a09bd553f0974376b550e0a7b801b083 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:15.641047 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05a27-bb5a-408a-92fc-6a328962a53a None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:15.641538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05a27-bb5a-408a-92fc-6a328962a53a None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:15.641647 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05a27-bb5a-408a-92fc-6a328962a53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:15.661036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-cbf05a27-bb5a-408a-92fc-6a328962a53a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:15.662245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cbf05a27-bb5a-408a-92fc-6a328962a53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:15.723220 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1316/2594] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:15.732879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:15.734971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:15.793326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8467b9-ef36-4e2f-bba2-dfaeb86e3fcc admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:15.797140 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b39a7-4ef0-43d7-ad9e-b78a02f303ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:15.798628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b39a7-4ef0-43d7-ad9e-b78a02f303ff service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:15.798958 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b39a7-4ef0-43d7-ad9e-b78a02f303ff service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:15.799243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b39a7-4ef0-43d7-ad9e-b78a02f303ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:15.809997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b39a7-4ef0-43d7-ad9e-b78a02f303ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:15.811441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b39a7-4ef0-43d7-ad9e-b78a02f303ff service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:15.822848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1279/2595] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:16.322340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf05a27-bb5a-408a-92fc-6a328962a53a None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:16.324602 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-cbf05a27-bb5a-408a-92fc-6a328962a53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:16.338038 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf05a27-bb5a-408a-92fc-6a328962a53a None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:16.343071 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:16.344016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:16.344509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:16.344987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] PATH_INFO: `/v3/projects/d5042734dfaf47d2b80bdd20c9daf3f0` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:16.346715 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5042734dfaf47d2b80bdd20c9daf3f0)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:16.349053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:16.479364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5042734dfaf47d2b80bdd20c9daf3f0', 'request_id': 'req-1251167e-8ddd-469c-a870-3b96964dfc2a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:16.549014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5042734dfaf47d2b80bdd20c9daf3f0', 'request_id': 'req-1251167e-8ddd-469c-a870-3b96964dfc2a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:16.556496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5042734dfaf47d2b80bdd20c9daf3f0', 'request_id': 'req-1251167e-8ddd-469c-a870-3b96964dfc2a'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:16.572754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] The token cache is being invalidate because project d5042734dfaf47d2b80bdd20c9daf3f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:16.573750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:16.574431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:16.580200 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1317/2596] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:16 2022] DELETE /identity/v3/projects/d5042734dfaf47d2b80bdd20c9daf3f0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:16.581205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b39a7-4ef0-43d7-ad9e-b78a02f303ff service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:16.596250 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b39a7-4ef0-43d7-ad9e-b78a02f303ff service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:16.623902 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:16.664224 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:16.664865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:16.665152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:16.665433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] PATH_INFO: `/v3/users/08454683053f4c00ad4b26505c706949` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:16.666702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08454683053f4c00ad4b26505c706949)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:16.668850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:17.306940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08454683053f4c00ad4b26505c706949'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.307752 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08454683053f4c00ad4b26505c706949'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.320475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08454683053f4c00ad4b26505c706949'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.411555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08454683053f4c00ad4b26505c706949'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.419982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08454683053f4c00ad4b26505c706949'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.426286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1280/2597] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:16 2022] DELETE /identity/v3/users/08454683053f4c00ad4b26505c706949 => generated 0 bytes in 846 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:18.339913 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:18.341700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:18.357511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251167e-8ddd-469c-a870-3b96964dfc2a admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:18.379479 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:18.419409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:18.420082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:18.420371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:18.420652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] PATH_INFO: `/v3/projects/9a0905ad363d4221896e7e8e904dfa4d` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:18.421830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0905ad363d4221896e7e8e904dfa4d)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:18.423751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:18.578814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0905ad363d4221896e7e8e904dfa4d', 'request_id': 'req-73fe59d4-2780-4685-a7fb-da4185d104a4'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.669375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0905ad363d4221896e7e8e904dfa4d', 'request_id': 'req-73fe59d4-2780-4685-a7fb-da4185d104a4'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.676211 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0905ad363d4221896e7e8e904dfa4d', 'request_id': 'req-73fe59d4-2780-4685-a7fb-da4185d104a4'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.688471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] The token cache is being invalidate because project 9a0905ad363d4221896e7e8e904dfa4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:18.688770 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.689629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.691592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1318/2598] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:18 2022] DELETE /identity/v3/projects/9a0905ad363d4221896e7e8e904dfa4d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:18.696123 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:18.697563 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:18.711571 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ee9ec-e845-4252-bbce-80b5d9fe0cf8 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:18.777903 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50d9cc-12a1-4922-8ebb-0db6cd31bfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:18.778653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50d9cc-12a1-4922-8ebb-0db6cd31bfe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:18.778993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50d9cc-12a1-4922-8ebb-0db6cd31bfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:18.779281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50d9cc-12a1-4922-8ebb-0db6cd31bfe8 admin admin] PATH_INFO: `/v3/users/61b92c8453a342b8891ab30355ca2b8d` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:18.780475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50d9cc-12a1-4922-8ebb-0db6cd31bfe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b92c8453a342b8891ab30355ca2b8d)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:18.782426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50d9cc-12a1-4922-8ebb-0db6cd31bfe8 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:18.957221 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:18.963674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9c50d9cc-12a1-4922-8ebb-0db6cd31bfe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b92c8453a342b8891ab30355ca2b8d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.964453 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9c50d9cc-12a1-4922-8ebb-0db6cd31bfe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b92c8453a342b8891ab30355ca2b8d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.979402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9c50d9cc-12a1-4922-8ebb-0db6cd31bfe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b92c8453a342b8891ab30355ca2b8d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.070208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9c50d9cc-12a1-4922-8ebb-0db6cd31bfe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b92c8453a342b8891ab30355ca2b8d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.081336 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9c50d9cc-12a1-4922-8ebb-0db6cd31bfe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b92c8453a342b8891ab30355ca2b8d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.088594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1281/2599] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:18 2022] DELETE /identity/v3/users/61b92c8453a342b8891ab30355ca2b8d => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:19.675594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:19.677093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:19.690319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe59d4-2780-4685-a7fb-da4185d104a4 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:19.749034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:19.749668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.749954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.750239 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] PATH_INFO: `/v3/projects/087001e6693e425d8d072ef6dcb78de0` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:19.751380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=087001e6693e425d8d072ef6dcb78de0)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:19.753413 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:19.863344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087001e6693e425d8d072ef6dcb78de0', 'request_id': 'req-92556caf-8ace-4691-af7b-a98f9a7944cd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.954414 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087001e6693e425d8d072ef6dcb78de0', 'request_id': 'req-92556caf-8ace-4691-af7b-a98f9a7944cd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.955792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:19.961994 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087001e6693e425d8d072ef6dcb78de0', 'request_id': 'req-92556caf-8ace-4691-af7b-a98f9a7944cd'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.974442 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] The token cache is being invalidate because project 087001e6693e425d8d072ef6dcb78de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:19.975301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.976026 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-92556caf-8ace-4691-af7b-a98f9a7944cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.977526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1319/2600] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:19 2022] DELETE /identity/v3/projects/087001e6693e425d8d072ef6dcb78de0 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:19.983566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed348-319f-4a3a-81d1-56df117fbdf0 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.984149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed348-319f-4a3a-81d1-56df117fbdf0 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.984497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed348-319f-4a3a-81d1-56df117fbdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.004310 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-75bed348-319f-4a3a-81d1-56df117fbdf0 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:20.005765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-75bed348-319f-4a3a-81d1-56df117fbdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:20.067379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1282/2601] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:20.075716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a490a8-b6e7-4d3d-a771-b87c9470aa52 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.076043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a490a8-b6e7-4d3d-a771-b87c9470aa52 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.076300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a490a8-b6e7-4d3d-a771-b87c9470aa52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.096682 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-61a490a8-b6e7-4d3d-a771-b87c9470aa52 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:20.097907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-61a490a8-b6e7-4d3d-a771-b87c9470aa52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:20.107119 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1320/2602] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:20.116604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bed348-319f-4a3a-81d1-56df117fbdf0 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.126426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bed348-319f-4a3a-81d1-56df117fbdf0 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.131677 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29f983-4af6-49f7-bf46-237e91eb4fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.132320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29f983-4af6-49f7-bf46-237e91eb4fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.132600 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29f983-4af6-49f7-bf46-237e91eb4fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.132875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29f983-4af6-49f7-bf46-237e91eb4fb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.133534 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29f983-4af6-49f7-bf46-237e91eb4fb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:20.133838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29f983-4af6-49f7-bf46-237e91eb4fb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.135444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29f983-4af6-49f7-bf46-237e91eb4fb4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.148082 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1283/2603] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:20.152408 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a490a8-b6e7-4d3d-a771-b87c9470aa52 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.161591 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a490a8-b6e7-4d3d-a771-b87c9470aa52 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.166363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a36e-229d-4a96-ae5a-f7706ba2e35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.166958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a36e-229d-4a96-ae5a-f7706ba2e35f admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.167244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a36e-229d-4a96-ae5a-f7706ba2e35f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.167525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a36e-229d-4a96-ae5a-f7706ba2e35f admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.168229 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8a36e-229d-4a96-ae5a-f7706ba2e35f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.170111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8a36e-229d-4a96-ae5a-f7706ba2e35f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.318854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1321/2604] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:20 2022] POST /identity/v3/projects => generated 399 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:20.323053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29f983-4af6-49f7-bf46-237e91eb4fb4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.333308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29f983-4af6-49f7-bf46-237e91eb4fb4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.393341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cecbb-0c5e-4708-8401-ec7dd7068035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.393993 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cecbb-0c5e-4708-8401-ec7dd7068035 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.394284 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cecbb-0c5e-4708-8401-ec7dd7068035 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.394587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cecbb-0c5e-4708-8401-ec7dd7068035 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.395211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cecbb-0c5e-4708-8401-ec7dd7068035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.397863 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cecbb-0c5e-4708-8401-ec7dd7068035 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.492151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1284/2605] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:20 2022] POST /identity/v3/users => generated 300 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:20.497033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a36e-229d-4a96-ae5a-f7706ba2e35f admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.508921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a36e-229d-4a96-ae5a-f7706ba2e35f admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.518934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8d479-498d-4f4d-9747-aeaf33d49627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.519677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8d479-498d-4f4d-9747-aeaf33d49627 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.520069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8d479-498d-4f4d-9747-aeaf33d49627 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.520443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8d479-498d-4f4d-9747-aeaf33d49627 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.521842 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8d479-498d-4f4d-9747-aeaf33d49627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.523526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8d479-498d-4f4d-9747-aeaf33d49627 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.534441 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1322/2606] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:20.538747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cecbb-0c5e-4708-8401-ec7dd7068035 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.549158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cecbb-0c5e-4708-8401-ec7dd7068035 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.554589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1199b-2a8a-4077-bbcb-10db6ca80dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.555194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1199b-2a8a-4077-bbcb-10db6ca80dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.555507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1199b-2a8a-4077-bbcb-10db6ca80dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.555783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1199b-2a8a-4077-bbcb-10db6ca80dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.556472 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a1199b-2a8a-4077-bbcb-10db6ca80dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.558075 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a1199b-2a8a-4077-bbcb-10db6ca80dc7 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.569085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1285/2607] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:20.573240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8d479-498d-4f4d-9747-aeaf33d49627 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.585633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8d479-498d-4f4d-9747-aeaf33d49627 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.591242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4ac23-9942-4fbd-b507-4974e33ae37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.592146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4ac23-9942-4fbd-b507-4974e33ae37b admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.592635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4ac23-9942-4fbd-b507-4974e33ae37b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.592964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4ac23-9942-4fbd-b507-4974e33ae37b admin admin] PATH_INFO: `/v3/projects/743bc840383b4f92b1134eedb87ac1f2/users/6d90e96579464011a9c28b6543b501ec/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.609758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4ac23-9942-4fbd-b507-4974e33ae37b admin admin] RBAC: Authorizing `identity:create_grant(project_id=743bc840383b4f92b1134eedb87ac1f2, user_id=6d90e96579464011a9c28b6543b501ec, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.612617 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4ac23-9942-4fbd-b507-4974e33ae37b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.709233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1323/2608] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:20 2022] PUT /identity/v3/projects/743bc840383b4f92b1134eedb87ac1f2/users/6d90e96579464011a9c28b6543b501ec/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:20.721450 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3635e12-3a33-4c96-882e-76800df54110 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.721741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3635e12-3a33-4c96-882e-76800df54110 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.722025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3635e12-3a33-4c96-882e-76800df54110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.750126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e3635e12-3a33-4c96-882e-76800df54110 None None] MFA Rules not processed for user `6d90e96579464011a9c28b6543b501ec`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:20.751352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e3635e12-3a33-4c96-882e-76800df54110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:20.764481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:20.820718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1286/2609] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:20 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:20.825256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4ac23-9942-4fbd-b507-4974e33ae37b admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.889319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4ac23-9942-4fbd-b507-4974e33ae37b admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.894076 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f535acc-6cc2-4df4-9b32-1ac6e8b551ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.894695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535acc-6cc2-4df4-9b32-1ac6e8b551ff admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.894960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535acc-6cc2-4df4-9b32-1ac6e8b551ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.895273 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535acc-6cc2-4df4-9b32-1ac6e8b551ff admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.896000 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f535acc-6cc2-4df4-9b32-1ac6e8b551ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.897902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f535acc-6cc2-4df4-9b32-1ac6e8b551ff admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:21.021659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1324/2610] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:20 2022] POST /identity/v3/projects => generated 399 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:21.022879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3635e12-3a33-4c96-882e-76800df54110 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:21.033719 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3635e12-3a33-4c96-882e-76800df54110 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:21.093210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2efb5-8c62-4ebd-ab6b-32a6dd849b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:21.093831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2efb5-8c62-4ebd-ab6b-32a6dd849b31 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:21.094115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2efb5-8c62-4ebd-ab6b-32a6dd849b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:21.094395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2efb5-8c62-4ebd-ab6b-32a6dd849b31 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:21.095026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2efb5-8c62-4ebd-ab6b-32a6dd849b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:21.096984 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2efb5-8c62-4ebd-ab6b-32a6dd849b31 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:21.241216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1287/2611] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:21 2022] POST /identity/v3/users => generated 306 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:21.245335 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f535acc-6cc2-4df4-9b32-1ac6e8b551ff admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:21.255650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f535acc-6cc2-4df4-9b32-1ac6e8b551ff admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:21.261342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab365e0d-13e1-4e3b-9a8b-7c1d860ecf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:21.261949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab365e0d-13e1-4e3b-9a8b-7c1d860ecf55 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:21.262238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab365e0d-13e1-4e3b-9a8b-7c1d860ecf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:21.262516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab365e0d-13e1-4e3b-9a8b-7c1d860ecf55 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:21.263167 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab365e0d-13e1-4e3b-9a8b-7c1d860ecf55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:21.264809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab365e0d-13e1-4e3b-9a8b-7c1d860ecf55 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:21.276550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1325/2612] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:21.280953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2efb5-8c62-4ebd-ab6b-32a6dd849b31 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:21.292306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2efb5-8c62-4ebd-ab6b-32a6dd849b31 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:21.297886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16097858-72e9-4398-9044-7c6520859627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:21.298340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16097858-72e9-4398-9044-7c6520859627 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:21.298651 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16097858-72e9-4398-9044-7c6520859627 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:21.298930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16097858-72e9-4398-9044-7c6520859627 admin admin] PATH_INFO: `/v3/projects/011575963e6f4ef7ab763acc767ff860/users/ddf4dc490638430784c99c110eb9c20d/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:21.316026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16097858-72e9-4398-9044-7c6520859627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011575963e6f4ef7ab763acc767ff860, user_id=ddf4dc490638430784c99c110eb9c20d, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:21.318750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16097858-72e9-4398-9044-7c6520859627 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:21.428826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1288/2613] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:21 2022] PUT /identity/v3/projects/011575963e6f4ef7ab763acc767ff860/users/ddf4dc490638430784c99c110eb9c20d/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:21.433042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab365e0d-13e1-4e3b-9a8b-7c1d860ecf55 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:21.443377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab365e0d-13e1-4e3b-9a8b-7c1d860ecf55 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:21.480440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40192950-45cc-4e6a-8ee5-5eef1da26ced None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:21.480698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40192950-45cc-4e6a-8ee5-5eef1da26ced None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:21.481023 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40192950-45cc-4e6a-8ee5-5eef1da26ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:21.501667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:21.502287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-40192950-45cc-4e6a-8ee5-5eef1da26ced None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:21.503830 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-40192950-45cc-4e6a-8ee5-5eef1da26ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:21.509864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644903db-d183-423a-8f94-221e7f211988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:21.514670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644903db-d183-423a-8f94-221e7f211988 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:21.514670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644903db-d183-423a-8f94-221e7f211988 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:21.514670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644903db-d183-423a-8f94-221e7f211988 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:21.514670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644903db-d183-423a-8f94-221e7f211988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:21.514670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644903db-d183-423a-8f94-221e7f211988 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:21.527907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1326/2614] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:21.530105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644903db-d183-423a-8f94-221e7f211988 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:21.541182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644903db-d183-423a-8f94-221e7f211988 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:21.545947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846b54c-1895-4b58-91ba-f09e931f72f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:21.546611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846b54c-1895-4b58-91ba-f09e931f72f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:21.546964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846b54c-1895-4b58-91ba-f09e931f72f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:21.547207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846b54c-1895-4b58-91ba-f09e931f72f4 admin admin] PATH_INFO: `/v3/projects/011575963e6f4ef7ab763acc767ff860/users/ddf4dc490638430784c99c110eb9c20d/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:21.549223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846b54c-1895-4b58-91ba-f09e931f72f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011575963e6f4ef7ab763acc767ff860, user_id=ddf4dc490638430784c99c110eb9c20d, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:21.551959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846b54c-1895-4b58-91ba-f09e931f72f4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:21.571815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1327/2615] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:21 2022] PUT /identity/v3/projects/011575963e6f4ef7ab763acc767ff860/users/ddf4dc490638430784c99c110eb9c20d/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:21.576057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5adb0c-57c2-4c55-a1bf-6cdca3a10c6c None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:21.576600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5adb0c-57c2-4c55-a1bf-6cdca3a10c6c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:21.577345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5adb0c-57c2-4c55-a1bf-6cdca3a10c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:21.578164 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1289/2616] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:21.596220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40192950-45cc-4e6a-8ee5-5eef1da26ced None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:21.597812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-40192950-45cc-4e6a-8ee5-5eef1da26ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:21.614463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-bd5adb0c-57c2-4c55-a1bf-6cdca3a10c6c None None] MFA Rules not processed for user `ddf4dc490638430784c99c110eb9c20d`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:21.616558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bd5adb0c-57c2-4c55-a1bf-6cdca3a10c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:21.643286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:21.690243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40192950-45cc-4e6a-8ee5-5eef1da26ced None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:21.702623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1328/2617] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:21.715408 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5adb0c-57c2-4c55-a1bf-6cdca3a10c6c None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:21.718862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bd5adb0c-57c2-4c55-a1bf-6cdca3a10c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:21.777003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a3283-52c8-4c19-b67e-69a5bb399ced service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:21.780227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a3283-52c8-4c19-b67e-69a5bb399ced service swift] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:21.780227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a3283-52c8-4c19-b67e-69a5bb399ced service swift] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:21.780227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a3283-52c8-4c19-b67e-69a5bb399ced service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:21.788720 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5adb0c-57c2-4c55-a1bf-6cdca3a10c6c None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:21.792766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9e755-d26e-4b08-872d-945200b5edd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:21.793389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9e755-d26e-4b08-872d-945200b5edd8 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:21.793678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9e755-d26e-4b08-872d-945200b5edd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:21.793942 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9e755-d26e-4b08-872d-945200b5edd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:21.804676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9e755-d26e-4b08-872d-945200b5edd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:21.806127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9e755-d26e-4b08-872d-945200b5edd8 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:21.817028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1329/2618] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:21 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:57:21.841619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a3283-52c8-4c19-b67e-69a5bb399ced service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:21.843331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a3283-52c8-4c19-b67e-69a5bb399ced service swift] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:21.852397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1290/2619] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:57:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 15:57:22.057585 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9e755-d26e-4b08-872d-945200b5edd8 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:22.067453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9e755-d26e-4b08-872d-945200b5edd8 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:22.076587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff1a8c-19aa-451c-aa11-ac1aca55d164 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:22.076587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff1a8c-19aa-451c-aa11-ac1aca55d164 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:22.076587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff1a8c-19aa-451c-aa11-ac1aca55d164 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:22.076587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff1a8c-19aa-451c-aa11-ac1aca55d164 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:22.143927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff1a8c-19aa-451c-aa11-ac1aca55d164 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:22.145393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff1a8c-19aa-451c-aa11-ac1aca55d164 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:22.159861 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1330/2620] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:22 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:28.100992 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a3283-52c8-4c19-b67e-69a5bb399ced service swift] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:28.165565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a3283-52c8-4c19-b67e-69a5bb399ced service swift] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:28.170711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:28.171278 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:28.171561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:28.171838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] PATH_INFO: `/v3/users/6d90e96579464011a9c28b6543b501ec` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:28.173018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d90e96579464011a9c28b6543b501ec)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:28.175014 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:28.188850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:28.348622 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d90e96579464011a9c28b6543b501ec'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:28.349610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d90e96579464011a9c28b6543b501ec'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:28.372593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d90e96579464011a9c28b6543b501ec'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:28.483448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d90e96579464011a9c28b6543b501ec'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:28.493086 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d90e96579464011a9c28b6543b501ec'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:28.499313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1291/2621] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:28 2022] DELETE /identity/v3/users/6d90e96579464011a9c28b6543b501ec => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:28.504101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538ff83-f718-4d5f-98fe-c6df3b41728f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:28.507436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538ff83-f718-4d5f-98fe-c6df3b41728f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:28.507436 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538ff83-f718-4d5f-98fe-c6df3b41728f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:28.513039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:28.525934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e538ff83-f718-4d5f-98fe-c6df3b41728f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:28.527174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e538ff83-f718-4d5f-98fe-c6df3b41728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:28.616547 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1331/2622] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:28.627410 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:28.629165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:28.707004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54b09c-c41c-4bbc-842e-94e4987a1b5f admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:28.712215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265d275-db6a-4d41-8605-4c00b013bac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:28.713412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265d275-db6a-4d41-8605-4c00b013bac6 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:28.713738 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265d275-db6a-4d41-8605-4c00b013bac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:28.714213 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265d275-db6a-4d41-8605-4c00b013bac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:28.726122 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265d275-db6a-4d41-8605-4c00b013bac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:28.727566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265d275-db6a-4d41-8605-4c00b013bac6 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:28.740643 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1292/2623] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:29.361439 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538ff83-f718-4d5f-98fe-c6df3b41728f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:29.363310 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e538ff83-f718-4d5f-98fe-c6df3b41728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:29.385313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538ff83-f718-4d5f-98fe-c6df3b41728f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:29.391523 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:29.392457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:29.392868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:29.393256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] PATH_INFO: `/v3/projects/743bc840383b4f92b1134eedb87ac1f2` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:29.394636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=743bc840383b4f92b1134eedb87ac1f2)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:29.397389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:29.547677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '743bc840383b4f92b1134eedb87ac1f2', 'request_id': 'req-8aff179a-c68e-434b-a522-73ba268f8437'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.656830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '743bc840383b4f92b1134eedb87ac1f2', 'request_id': 'req-8aff179a-c68e-434b-a522-73ba268f8437'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.664657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '743bc840383b4f92b1134eedb87ac1f2', 'request_id': 'req-8aff179a-c68e-434b-a522-73ba268f8437'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.678878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] The token cache is being invalidate because project 743bc840383b4f92b1134eedb87ac1f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:29.685814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.685814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.685814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1332/2624] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:29 2022] DELETE /identity/v3/projects/743bc840383b4f92b1134eedb87ac1f2 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:29.686117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265d275-db6a-4d41-8605-4c00b013bac6 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:29.702949 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265d275-db6a-4d41-8605-4c00b013bac6 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:29.785189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934439a-1c4d-481c-afb7-cb5395b7527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:29.785936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934439a-1c4d-481c-afb7-cb5395b7527b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:29.786280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934439a-1c4d-481c-afb7-cb5395b7527b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:29.786567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934439a-1c4d-481c-afb7-cb5395b7527b admin admin] PATH_INFO: `/v3/users/ddf4dc490638430784c99c110eb9c20d` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:29.788333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934439a-1c4d-481c-afb7-cb5395b7527b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddf4dc490638430784c99c110eb9c20d)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:29.794127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934439a-1c4d-481c-afb7-cb5395b7527b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:29.972130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:29.982735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e934439a-1c4d-481c-afb7-cb5395b7527b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddf4dc490638430784c99c110eb9c20d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.983628 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e934439a-1c4d-481c-afb7-cb5395b7527b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddf4dc490638430784c99c110eb9c20d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.999467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e934439a-1c4d-481c-afb7-cb5395b7527b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddf4dc490638430784c99c110eb9c20d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:30.062588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e934439a-1c4d-481c-afb7-cb5395b7527b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddf4dc490638430784c99c110eb9c20d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:30.072226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-e934439a-1c4d-481c-afb7-cb5395b7527b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddf4dc490638430784c99c110eb9c20d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:30.079740 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1293/2625] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:29 2022] DELETE /identity/v3/users/ddf4dc490638430784c99c110eb9c20d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:30.689642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:30.691420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:30.713454 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff179a-c68e-434b-a522-73ba268f8437 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:30.784914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:30.786066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:30.786664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:30.787319 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] PATH_INFO: `/v3/projects/011575963e6f4ef7ab763acc767ff860` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:30.789948 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=011575963e6f4ef7ab763acc767ff860)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:30.791930 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:30.902245 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:30.916172 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011575963e6f4ef7ab763acc767ff860', 'request_id': 'req-0e5e161e-a809-417b-94af-aaace0243f2f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:31.454923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011575963e6f4ef7ab763acc767ff860', 'request_id': 'req-0e5e161e-a809-417b-94af-aaace0243f2f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:31.462005 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011575963e6f4ef7ab763acc767ff860', 'request_id': 'req-0e5e161e-a809-417b-94af-aaace0243f2f'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:31.472701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] The token cache is being invalidate because project 011575963e6f4ef7ab763acc767ff860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:31.473033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:31.473689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0e5e161e-a809-417b-94af-aaace0243f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:31.475695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1333/2626] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:30 2022] DELETE /identity/v3/projects/011575963e6f4ef7ab763acc767ff860 => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:31.480440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ad782-aaec-4da4-9ffb-5a44426ff876 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.480901 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ad782-aaec-4da4-9ffb-5a44426ff876 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.481026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ad782-aaec-4da4-9ffb-5a44426ff876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.501608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-844ad782-aaec-4da4-9ffb-5a44426ff876 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:31.502815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-844ad782-aaec-4da4-9ffb-5a44426ff876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:31.571978 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1294/2627] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:31.580519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d174b-2631-4dc3-9608-33bce2823931 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.580966 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d174b-2631-4dc3-9608-33bce2823931 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.581260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d174b-2631-4dc3-9608-33bce2823931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.601938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-8a4d174b-2631-4dc3-9608-33bce2823931 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:31.603169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8a4d174b-2631-4dc3-9608-33bce2823931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:31.612506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1334/2628] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:31.621539 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ad782-aaec-4da4-9ffb-5a44426ff876 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:31.632100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ad782-aaec-4da4-9ffb-5a44426ff876 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:31.637025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a9d51-0653-4882-9394-c1be052d761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:31.637646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a9d51-0653-4882-9394-c1be052d761d admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.637934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a9d51-0653-4882-9394-c1be052d761d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.638244 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a9d51-0653-4882-9394-c1be052d761d admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.638907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a9d51-0653-4882-9394-c1be052d761d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:31.639215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a9d51-0653-4882-9394-c1be052d761d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:31.640858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a9d51-0653-4882-9394-c1be052d761d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:31.656286 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1295/2629] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:31.660904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d174b-2631-4dc3-9608-33bce2823931 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:31.670881 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d174b-2631-4dc3-9608-33bce2823931 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:31.675618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb968e5-6fd0-4d67-8ae2-0f11cb87dcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:31.676240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb968e5-6fd0-4d67-8ae2-0f11cb87dcb3 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.676525 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb968e5-6fd0-4d67-8ae2-0f11cb87dcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.676803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb968e5-6fd0-4d67-8ae2-0f11cb87dcb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.678424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb968e5-6fd0-4d67-8ae2-0f11cb87dcb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:31.680272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb968e5-6fd0-4d67-8ae2-0f11cb87dcb3 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:31.825363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1335/2630] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:31 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:31.830334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a9d51-0653-4882-9394-c1be052d761d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:31.843709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a9d51-0653-4882-9394-c1be052d761d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:31.936413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5a7c2-e28a-49ba-88cc-ea2e6c285cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:31.937188 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5a7c2-e28a-49ba-88cc-ea2e6c285cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.937501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5a7c2-e28a-49ba-88cc-ea2e6c285cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.937754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5a7c2-e28a-49ba-88cc-ea2e6c285cc1 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.938390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5a7c2-e28a-49ba-88cc-ea2e6c285cc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:31.940391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5a7c2-e28a-49ba-88cc-ea2e6c285cc1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:32.054943 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1296/2631] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:31 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:32.056569 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb968e5-6fd0-4d67-8ae2-0f11cb87dcb3 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:32.069871 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb968e5-6fd0-4d67-8ae2-0f11cb87dcb3 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:32.074750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355aa8a-d733-4609-beeb-efc6c825cbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:32.075420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355aa8a-d733-4609-beeb-efc6c825cbbd admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:32.075653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355aa8a-d733-4609-beeb-efc6c825cbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:32.076036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355aa8a-d733-4609-beeb-efc6c825cbbd admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:32.076694 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355aa8a-d733-4609-beeb-efc6c825cbbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:32.078696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355aa8a-d733-4609-beeb-efc6c825cbbd admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:32.094684 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1336/2632] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:32.097375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5a7c2-e28a-49ba-88cc-ea2e6c285cc1 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:32.109473 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5a7c2-e28a-49ba-88cc-ea2e6c285cc1 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:32.114981 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09f0b9-bff8-4c0c-8b45-b0b73a113839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:32.115633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09f0b9-bff8-4c0c-8b45-b0b73a113839 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:32.115947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09f0b9-bff8-4c0c-8b45-b0b73a113839 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:32.116228 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09f0b9-bff8-4c0c-8b45-b0b73a113839 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:32.116880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09f0b9-bff8-4c0c-8b45-b0b73a113839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:32.118900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09f0b9-bff8-4c0c-8b45-b0b73a113839 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:32.132577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1297/2633] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:32.137039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355aa8a-d733-4609-beeb-efc6c825cbbd admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:32.151588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355aa8a-d733-4609-beeb-efc6c825cbbd admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:32.156519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd155e2-7c82-4c60-93ca-e24d6901b41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:32.157242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd155e2-7c82-4c60-93ca-e24d6901b41c admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:32.157545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd155e2-7c82-4c60-93ca-e24d6901b41c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:32.157830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd155e2-7c82-4c60-93ca-e24d6901b41c admin admin] PATH_INFO: `/v3/projects/c5f8087306de4ddf8dea4db4ca587646/users/d560eaf6bed741b5a60f0b24e2dd559d/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:32.175934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd155e2-7c82-4c60-93ca-e24d6901b41c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f8087306de4ddf8dea4db4ca587646, user_id=d560eaf6bed741b5a60f0b24e2dd559d, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:32.178962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd155e2-7c82-4c60-93ca-e24d6901b41c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:32.251843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1337/2634] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:32 2022] PUT /identity/v3/projects/c5f8087306de4ddf8dea4db4ca587646/users/d560eaf6bed741b5a60f0b24e2dd559d/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:32.263626 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a97d01-c5bd-4765-950f-3c483b989b0f None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:32.264030 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a97d01-c5bd-4765-950f-3c483b989b0f None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:32.264393 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a97d01-c5bd-4765-950f-3c483b989b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:32.293999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-e4a97d01-c5bd-4765-950f-3c483b989b0f None None] MFA Rules not processed for user `d560eaf6bed741b5a60f0b24e2dd559d`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:32.295223 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-e4a97d01-c5bd-4765-950f-3c483b989b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:32.301032 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:32.368411 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1298/2635] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:32.380809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd155e2-7c82-4c60-93ca-e24d6901b41c admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:32.382844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bdd155e2-7c82-4c60-93ca-e24d6901b41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:32.446673 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd155e2-7c82-4c60-93ca-e24d6901b41c admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:32.450964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002c0c4-219b-4376-9446-4eef727bba16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:32.452181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002c0c4-219b-4376-9446-4eef727bba16 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:32.452934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002c0c4-219b-4376-9446-4eef727bba16 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:32.453259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002c0c4-219b-4376-9446-4eef727bba16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:32.462940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002c0c4-219b-4376-9446-4eef727bba16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:32.464417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002c0c4-219b-4376-9446-4eef727bba16 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:32.473134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1338/2636] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:32.495226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a97d01-c5bd-4765-950f-3c483b989b0f None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:32.558715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a97d01-c5bd-4765-950f-3c483b989b0f None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:32.563477 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:32.564149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:32.564438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:32.564720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] PATH_INFO: `/v3/users/d560eaf6bed741b5a60f0b24e2dd559d` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:32.565869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d560eaf6bed741b5a60f0b24e2dd559d)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:32.568382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:32.898408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd560eaf6bed741b5a60f0b24e2dd559d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:32.899001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd560eaf6bed741b5a60f0b24e2dd559d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:32.911141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd560eaf6bed741b5a60f0b24e2dd559d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:32.997584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd560eaf6bed741b5a60f0b24e2dd559d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:33.005953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd560eaf6bed741b5a60f0b24e2dd559d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:33.012366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1299/2637] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:32 2022] DELETE /identity/v3/users/d560eaf6bed741b5a60f0b24e2dd559d => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:33.017043 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb20f00-acb3-4827-8e12-0bb258056796 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:33.021905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb20f00-acb3-4827-8e12-0bb258056796 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:33.021905 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb20f00-acb3-4827-8e12-0bb258056796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:33.038024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ddb20f00-acb3-4827-8e12-0bb258056796 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:33.039262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ddb20f00-acb3-4827-8e12-0bb258056796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:33.091136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:33.101945 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1339/2638] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:33.112406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:33.113875 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:33.146079 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:33.172977 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c48da-e80b-4bf8-bd0e-804e2fef2d33 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:33.176512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd35340-4a64-43cd-98e3-958a3467e852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:33.177711 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd35340-4a64-43cd-98e3-958a3467e852 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:33.177996 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd35340-4a64-43cd-98e3-958a3467e852 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:33.178273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd35340-4a64-43cd-98e3-958a3467e852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:33.187530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd35340-4a64-43cd-98e3-958a3467e852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:33.188986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd35340-4a64-43cd-98e3-958a3467e852 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:33.198715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1300/2639] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:33.630821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb20f00-acb3-4827-8e12-0bb258056796 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:33.633367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ddb20f00-acb3-4827-8e12-0bb258056796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:33.650331 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb20f00-acb3-4827-8e12-0bb258056796 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:33.655061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:33.655698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:33.656022 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:33.656304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] PATH_INFO: `/v3/projects/c5f8087306de4ddf8dea4db4ca587646` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:33.657439 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f8087306de4ddf8dea4db4ca587646)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:33.659336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:33.790067 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f8087306de4ddf8dea4db4ca587646', 'request_id': 'req-5031750e-7236-4ccc-9ff8-537b1119d38d'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:33.868605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f8087306de4ddf8dea4db4ca587646', 'request_id': 'req-5031750e-7236-4ccc-9ff8-537b1119d38d'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:33.875792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f8087306de4ddf8dea4db4ca587646', 'request_id': 'req-5031750e-7236-4ccc-9ff8-537b1119d38d'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:33.887426 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] The token cache is being invalidate because project c5f8087306de4ddf8dea4db4ca587646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:33.887724 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:33.888463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-5031750e-7236-4ccc-9ff8-537b1119d38d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:33.890126 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1340/2640] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:33 2022] DELETE /identity/v3/projects/c5f8087306de4ddf8dea4db4ca587646 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:33.895003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff62c45-adc4-4a77-b4ec-ad049d8b45f9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:33.899883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff62c45-adc4-4a77-b4ec-ad049d8b45f9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:33.899883 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff62c45-adc4-4a77-b4ec-ad049d8b45f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:33.918263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-eff62c45-adc4-4a77-b4ec-ad049d8b45f9 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:33.919461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-eff62c45-adc4-4a77-b4ec-ad049d8b45f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:33.982470 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1301/2641] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:33.990959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686d6f0-0cea-4aa9-8048-4150e11b623d None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:33.991252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686d6f0-0cea-4aa9-8048-4150e11b623d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:33.991700 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686d6f0-0cea-4aa9-8048-4150e11b623d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.011405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-3686d6f0-0cea-4aa9-8048-4150e11b623d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:34.012670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-3686d6f0-0cea-4aa9-8048-4150e11b623d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:34.021506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1341/2642] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:34.025720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff62c45-adc4-4a77-b4ec-ad049d8b45f9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:34.035885 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff62c45-adc4-4a77-b4ec-ad049d8b45f9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:34.041120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5b6a6-2f29-4b2d-abdb-97b06c464340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:34.042945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5b6a6-2f29-4b2d-abdb-97b06c464340 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.042945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5b6a6-2f29-4b2d-abdb-97b06c464340 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.042945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5b6a6-2f29-4b2d-abdb-97b06c464340 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.042945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5b6a6-2f29-4b2d-abdb-97b06c464340 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:34.043289 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5b6a6-2f29-4b2d-abdb-97b06c464340 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:34.044824 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5b6a6-2f29-4b2d-abdb-97b06c464340 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:34.057462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1302/2643] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:34.061985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686d6f0-0cea-4aa9-8048-4150e11b623d None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:34.071416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686d6f0-0cea-4aa9-8048-4150e11b623d None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:34.076194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c57111-67c6-4f19-96b3-66f8dc43050e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:34.076878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c57111-67c6-4f19-96b3-66f8dc43050e admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.077633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c57111-67c6-4f19-96b3-66f8dc43050e admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.077827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c57111-67c6-4f19-96b3-66f8dc43050e admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.078558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c57111-67c6-4f19-96b3-66f8dc43050e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:34.080474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c57111-67c6-4f19-96b3-66f8dc43050e admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:34.183746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1342/2644] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:34 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:34.187821 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5b6a6-2f29-4b2d-abdb-97b06c464340 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:34.199577 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5b6a6-2f29-4b2d-abdb-97b06c464340 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:34.260986 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bbb43-6963-4667-8042-194fd806f633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:34.261609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bbb43-6963-4667-8042-194fd806f633 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.261929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bbb43-6963-4667-8042-194fd806f633 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.262212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bbb43-6963-4667-8042-194fd806f633 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.262849 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644bbb43-6963-4667-8042-194fd806f633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:34.264803 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644bbb43-6963-4667-8042-194fd806f633 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:34.404868 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1303/2645] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:34 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:34.412067 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c57111-67c6-4f19-96b3-66f8dc43050e admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:34.419887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c57111-67c6-4f19-96b3-66f8dc43050e admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:34.424512 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a534d-10bb-4e46-8777-c359e82510d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:34.425106 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a534d-10bb-4e46-8777-c359e82510d8 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.425393 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a534d-10bb-4e46-8777-c359e82510d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.425692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a534d-10bb-4e46-8777-c359e82510d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.426318 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a534d-10bb-4e46-8777-c359e82510d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:34.428600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a534d-10bb-4e46-8777-c359e82510d8 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:34.438533 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1343/2646] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:34.442635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bbb43-6963-4667-8042-194fd806f633 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:34.453997 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bbb43-6963-4667-8042-194fd806f633 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:34.458582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805c2c5-222b-406c-b087-e8fc14ba636a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:34.459201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805c2c5-222b-406c-b087-e8fc14ba636a admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.459478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805c2c5-222b-406c-b087-e8fc14ba636a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.459753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805c2c5-222b-406c-b087-e8fc14ba636a admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.460422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805c2c5-222b-406c-b087-e8fc14ba636a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:34.461991 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805c2c5-222b-406c-b087-e8fc14ba636a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:34.472410 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1304/2647] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:34.476947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a534d-10bb-4e46-8777-c359e82510d8 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:34.487163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a534d-10bb-4e46-8777-c359e82510d8 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:34.492428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa862a5-cff7-4041-87be-4e5201f9bef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:34.493104 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa862a5-cff7-4041-87be-4e5201f9bef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.493392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa862a5-cff7-4041-87be-4e5201f9bef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.493679 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa862a5-cff7-4041-87be-4e5201f9bef2 admin admin] PATH_INFO: `/v3/projects/63ae2906c279490698a9dda1baa7ba82/users/305865d0a3194d35a2df419b2c284a3a/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.508237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa862a5-cff7-4041-87be-4e5201f9bef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63ae2906c279490698a9dda1baa7ba82, user_id=305865d0a3194d35a2df419b2c284a3a, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:34.510887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa862a5-cff7-4041-87be-4e5201f9bef2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:34.616733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1344/2648] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:34 2022] PUT /identity/v3/projects/63ae2906c279490698a9dda1baa7ba82/users/305865d0a3194d35a2df419b2c284a3a/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:34.627064 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ac8c0-59e6-404a-8378-bca1ca7dd5f8 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.627379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ac8c0-59e6-404a-8378-bca1ca7dd5f8 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.627658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ac8c0-59e6-404a-8378-bca1ca7dd5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.657617 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-549ac8c0-59e6-404a-8378-bca1ca7dd5f8 None None] MFA Rules not processed for user `305865d0a3194d35a2df419b2c284a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:34.658773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-549ac8c0-59e6-404a-8378-bca1ca7dd5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:34.683215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:34.726463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1305/2649] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:34.731152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa862a5-cff7-4041-87be-4e5201f9bef2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:34.804982 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa862a5-cff7-4041-87be-4e5201f9bef2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:34.809912 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c19a4-c780-4e09-9a91-c290b9fd8796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:34.810515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c19a4-c780-4e09-9a91-c290b9fd8796 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.810803 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c19a4-c780-4e09-9a91-c290b9fd8796 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.811086 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c19a4-c780-4e09-9a91-c290b9fd8796 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.811726 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c19a4-c780-4e09-9a91-c290b9fd8796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:34.813676 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c19a4-c780-4e09-9a91-c290b9fd8796 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:34.902651 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1345/2650] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:34 2022] POST /identity/v3/projects => generated 369 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:34.906888 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ac8c0-59e6-404a-8378-bca1ca7dd5f8 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:34.919429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ac8c0-59e6-404a-8378-bca1ca7dd5f8 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:34.983489 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e92080-5163-42b3-ae32-2f8b0dd24964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:34.984157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e92080-5163-42b3-ae32-2f8b0dd24964 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.984442 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e92080-5163-42b3-ae32-2f8b0dd24964 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.984717 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e92080-5163-42b3-ae32-2f8b0dd24964 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.985378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e92080-5163-42b3-ae32-2f8b0dd24964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:34.987312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e92080-5163-42b3-ae32-2f8b0dd24964 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:35.092199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1306/2651] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:34 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:35.096252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c19a4-c780-4e09-9a91-c290b9fd8796 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:35.103058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:35.109074 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c19a4-c780-4e09-9a91-c290b9fd8796 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:35.114080 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49dd13-932b-4138-a07b-51a6b4563aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:35.115232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49dd13-932b-4138-a07b-51a6b4563aea admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:35.115511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49dd13-932b-4138-a07b-51a6b4563aea admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:35.115793 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49dd13-932b-4138-a07b-51a6b4563aea admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:35.116514 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49dd13-932b-4138-a07b-51a6b4563aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:35.118150 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49dd13-932b-4138-a07b-51a6b4563aea admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:35.130811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1346/2652] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:35.135156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e92080-5163-42b3-ae32-2f8b0dd24964 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:35.145962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e92080-5163-42b3-ae32-2f8b0dd24964 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:35.150660 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621036e-bfae-4169-8c66-8fdbb518646b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:35.151379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621036e-bfae-4169-8c66-8fdbb518646b admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:35.151668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621036e-bfae-4169-8c66-8fdbb518646b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:35.152160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621036e-bfae-4169-8c66-8fdbb518646b admin admin] PATH_INFO: `/v3/projects/861f3394e7e24b7697f45280f38bb48c/users/ca568fd287fd414c8a88a8eccfc9d113/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:35.168865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621036e-bfae-4169-8c66-8fdbb518646b admin admin] RBAC: Authorizing `identity:create_grant(project_id=861f3394e7e24b7697f45280f38bb48c, user_id=ca568fd287fd414c8a88a8eccfc9d113, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:35.171630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621036e-bfae-4169-8c66-8fdbb518646b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:35.244252 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1307/2653] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:35 2022] PUT /identity/v3/projects/861f3394e7e24b7697f45280f38bb48c/users/ca568fd287fd414c8a88a8eccfc9d113/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:35.248324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49dd13-932b-4138-a07b-51a6b4563aea admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:35.258376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49dd13-932b-4138-a07b-51a6b4563aea admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:35.318724 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0f075-40ec-4199-b52d-dd94fe8ef214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:35.319336 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0f075-40ec-4199-b52d-dd94fe8ef214 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:35.319623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0f075-40ec-4199-b52d-dd94fe8ef214 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:35.319993 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0f075-40ec-4199-b52d-dd94fe8ef214 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:35.320582 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0f075-40ec-4199-b52d-dd94fe8ef214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:35.322187 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0f075-40ec-4199-b52d-dd94fe8ef214 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:35.336007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1347/2654] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:35.339852 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621036e-bfae-4169-8c66-8fdbb518646b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:35.350656 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621036e-bfae-4169-8c66-8fdbb518646b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:35.355321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38fafc-31f9-43c6-84eb-4c2454955547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:35.356009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38fafc-31f9-43c6-84eb-4c2454955547 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:35.356293 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38fafc-31f9-43c6-84eb-4c2454955547 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:35.356565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38fafc-31f9-43c6-84eb-4c2454955547 admin admin] PATH_INFO: `/v3/projects/861f3394e7e24b7697f45280f38bb48c/users/ca568fd287fd414c8a88a8eccfc9d113/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:35.358471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38fafc-31f9-43c6-84eb-4c2454955547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=861f3394e7e24b7697f45280f38bb48c, user_id=ca568fd287fd414c8a88a8eccfc9d113, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:35.361156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38fafc-31f9-43c6-84eb-4c2454955547 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:35.371697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1308/2655] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:35 2022] PUT /identity/v3/projects/861f3394e7e24b7697f45280f38bb48c/users/ca568fd287fd414c8a88a8eccfc9d113/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:35.445081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31347c3f-5d45-4f58-a3da-50bfbdd43ffb None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:35.445375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31347c3f-5d45-4f58-a3da-50bfbdd43ffb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:35.445662 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31347c3f-5d45-4f58-a3da-50bfbdd43ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:35.475798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-31347c3f-5d45-4f58-a3da-50bfbdd43ffb None None] MFA Rules not processed for user `ca568fd287fd414c8a88a8eccfc9d113`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:35.477070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-31347c3f-5d45-4f58-a3da-50bfbdd43ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:35.548084 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1348/2656] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:35.559174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38fafc-31f9-43c6-84eb-4c2454955547 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:35.560934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ef38fafc-31f9-43c6-84eb-4c2454955547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:35.632620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38fafc-31f9-43c6-84eb-4c2454955547 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:35.636353 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced96dd-b193-4a94-bd68-7bde79363a5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:35.636971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced96dd-b193-4a94-bd68-7bde79363a5b service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:35.641377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced96dd-b193-4a94-bd68-7bde79363a5b service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:35.641377 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced96dd-b193-4a94-bd68-7bde79363a5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:35.668784 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:35.695407 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced96dd-b193-4a94-bd68-7bde79363a5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:35.696803 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced96dd-b193-4a94-bd68-7bde79363a5b service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:35.706089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1309/2657] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:38.343972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311cb6d-0d23-456b-a214-96cf08adc094 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:38.344550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311cb6d-0d23-456b-a214-96cf08adc094 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:38.344550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311cb6d-0d23-456b-a214-96cf08adc094 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:38.347018 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1349/2658] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 15:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:57:38.352141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199140-266d-4c61-8acd-4074b82ec882 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:38.352664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199140-266d-4c61-8acd-4074b82ec882 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:38.352766 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199140-266d-4c61-8acd-4074b82ec882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:38.373058 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-3d199140-266d-4c61-8acd-4074b82ec882 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:38.374368 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3d199140-266d-4c61-8acd-4074b82ec882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:38.447373 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1310/2659] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 15:57:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:38.458650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311cb6d-0d23-456b-a214-96cf08adc094 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:38.460379 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c311cb6d-0d23-456b-a214-96cf08adc094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:38.521193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311cb6d-0d23-456b-a214-96cf08adc094 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:38.524916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5331435-40ae-45b0-a654-8530517d6bb9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cef193a54eff44f2b53716e3ec1a29cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:38.525524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5331435-40ae-45b0-a654-8530517d6bb9 service glance] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:38.526866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5331435-40ae-45b0-a654-8530517d6bb9 service glance] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:38.526866 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5331435-40ae-45b0-a654-8530517d6bb9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:38.537235 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5331435-40ae-45b0-a654-8530517d6bb9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:38.538675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5331435-40ae-45b0-a654-8530517d6bb9 service glance] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:38.549904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1350/2660] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:38.824323 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a928-d0d5-4171-8a27-a124b09a02ac None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:38.824524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a928-d0d5-4171-8a27-a124b09a02ac None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:38.824784 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a928-d0d5-4171-8a27-a124b09a02ac None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:38.826149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1311/2661] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 15:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:57:38.832382 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7645c-8883-4186-a308-d54d3d5945b6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:38.832678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7645c-8883-4186-a308-d54d3d5945b6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:38.833465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7645c-8883-4186-a308-d54d3d5945b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:38.854945 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-30c7645c-8883-4186-a308-d54d3d5945b6 None None] MFA Rules not processed for user `8c6cad18359143868276eb9d79a3fd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:38.856255 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-30c7645c-8883-4186-a308-d54d3d5945b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:38.858869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752a928-d0d5-4171-8a27-a124b09a02ac None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:38.866910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:38.912404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1351/2662] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 15:57:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:38.925118 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7645c-8883-4186-a308-d54d3d5945b6 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:38.926698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-30c7645c-8883-4186-a308-d54d3d5945b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:38.930732 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752a928-d0d5-4171-8a27-a124b09a02ac None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:38.936439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:38.937077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:38.937355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:38.937625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] PATH_INFO: `/v3/users/305865d0a3194d35a2df419b2c284a3a` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:38.938775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=305865d0a3194d35a2df419b2c284a3a)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:38.940710 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:38.941768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7645c-8883-4186-a308-d54d3d5945b6 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:38.946443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dc6f6-878b-4ec2-8dd0-3ce5e54dd23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:38.947093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dc6f6-878b-4ec2-8dd0-3ce5e54dd23b service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:38.947377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dc6f6-878b-4ec2-8dd0-3ce5e54dd23b service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:38.947658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dc6f6-878b-4ec2-8dd0-3ce5e54dd23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:38.955732 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2dc6f6-878b-4ec2-8dd0-3ce5e54dd23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:38.957450 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2dc6f6-878b-4ec2-8dd0-3ce5e54dd23b service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:38.966687 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1352/2663] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:39.052407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ace2e-e9ae-469d-ab96-d717a0df022d None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:39.052721 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ace2e-e9ae-469d-ab96-d717a0df022d None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:39.053001 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ace2e-e9ae-469d-ab96-d717a0df022d None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:39.054425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1353/2664] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 15:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:57:39.059780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561f78a-cc6d-4d72-828c-c7f64d9892ca None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:39.060189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561f78a-cc6d-4d72-828c-c7f64d9892ca None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:39.060481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561f78a-cc6d-4d72-828c-c7f64d9892ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:39.081189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-8561f78a-cc6d-4d72-828c-c7f64d9892ca None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:39.082329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-8561f78a-cc6d-4d72-828c-c7f64d9892ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:39.092936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1354/2665] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 15:57:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:39.103339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561f78a-cc6d-4d72-828c-c7f64d9892ca None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:39.112854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561f78a-cc6d-4d72-828c-c7f64d9892ca None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:39.116572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7756e-2478-4584-88bf-9ec36779af20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:39.117177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7756e-2478-4584-88bf-9ec36779af20 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:39.117471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7756e-2478-4584-88bf-9ec36779af20 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:39.117746 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7756e-2478-4584-88bf-9ec36779af20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:39.128169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7756e-2478-4584-88bf-9ec36779af20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:39.129595 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7756e-2478-4584-88bf-9ec36779af20 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:39.140951 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1355/2666] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:57:39.185584 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305865d0a3194d35a2df419b2c284a3a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:39.186279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305865d0a3194d35a2df419b2c284a3a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:39.199272 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305865d0a3194d35a2df419b2c284a3a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:39.201019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:39.267059 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305865d0a3194d35a2df419b2c284a3a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:39.277601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305865d0a3194d35a2df419b2c284a3a'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:39.284065 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1312/2667] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:38 2022] DELETE /identity/v3/users/305865d0a3194d35a2df419b2c284a3a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:39.289084 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d71ba-96a8-428c-833d-0682fdc51eb6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:39.289383 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d71ba-96a8-428c-833d-0682fdc51eb6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:39.289669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d71ba-96a8-428c-833d-0682fdc51eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:39.309409 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-bd9d71ba-96a8-428c-833d-0682fdc51eb6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:39.310640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bd9d71ba-96a8-428c-833d-0682fdc51eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:39.375227 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1356/2668] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:39.382243 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:39.383693 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:39.445755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0db55-83ff-48d5-9b37-2d0de7bec656 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:39.449823 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65aae8-f1fe-45ba-97ef-684a9b2f2981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:39.450565 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65aae8-f1fe-45ba-97ef-684a9b2f2981 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:39.450850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65aae8-f1fe-45ba-97ef-684a9b2f2981 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:39.451137 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65aae8-f1fe-45ba-97ef-684a9b2f2981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:39.462737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65aae8-f1fe-45ba-97ef-684a9b2f2981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:39.464416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65aae8-f1fe-45ba-97ef-684a9b2f2981 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:39.476562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1313/2669] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:39.943938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d71ba-96a8-428c-833d-0682fdc51eb6 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:39.945410 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bd9d71ba-96a8-428c-833d-0682fdc51eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:39.959315 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d71ba-96a8-428c-833d-0682fdc51eb6 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:39.964136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:39.966011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:39.966011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:39.966011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] PATH_INFO: `/v3/projects/63ae2906c279490698a9dda1baa7ba82` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:39.966799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63ae2906c279490698a9dda1baa7ba82)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:39.968867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:40.112338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63ae2906c279490698a9dda1baa7ba82', 'request_id': 'req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.176027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63ae2906c279490698a9dda1baa7ba82', 'request_id': 'req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.183292 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63ae2906c279490698a9dda1baa7ba82', 'request_id': 'req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.194439 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] The token cache is being invalidate because project 63ae2906c279490698a9dda1baa7ba82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:40.194439 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.195095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.196937 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1357/2670] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:39 2022] DELETE /identity/v3/projects/63ae2906c279490698a9dda1baa7ba82 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:40.200959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65aae8-f1fe-45ba-97ef-684a9b2f2981 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:40.212507 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65aae8-f1fe-45ba-97ef-684a9b2f2981 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:40.271814 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424a7c9-8198-4924-9199-d6ce1dfd2a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:40.272484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424a7c9-8198-4924-9199-d6ce1dfd2a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:40.272769 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424a7c9-8198-4924-9199-d6ce1dfd2a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:40.273048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424a7c9-8198-4924-9199-d6ce1dfd2a7b admin admin] PATH_INFO: `/v3/users/ca568fd287fd414c8a88a8eccfc9d113` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:40.274229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424a7c9-8198-4924-9199-d6ce1dfd2a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca568fd287fd414c8a88a8eccfc9d113)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:40.276167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424a7c9-8198-4924-9199-d6ce1dfd2a7b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:40.453234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a424a7c9-8198-4924-9199-d6ce1dfd2a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca568fd287fd414c8a88a8eccfc9d113'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.453915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a424a7c9-8198-4924-9199-d6ce1dfd2a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca568fd287fd414c8a88a8eccfc9d113'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.466288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a424a7c9-8198-4924-9199-d6ce1dfd2a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca568fd287fd414c8a88a8eccfc9d113'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.512696 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a424a7c9-8198-4924-9199-d6ce1dfd2a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca568fd287fd414c8a88a8eccfc9d113'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.520109 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-a424a7c9-8198-4924-9199-d6ce1dfd2a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca568fd287fd414c8a88a8eccfc9d113'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.526002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1314/2671] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:40 2022] DELETE /identity/v3/users/ca568fd287fd414c8a88a8eccfc9d113 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:41.018034 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:41.019587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:41.033128 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fea25c-9cd0-4d2c-9f23-c021d56d85e7 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:41.046639 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:41.095452 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:41.096135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:41.096419 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:41.096698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] PATH_INFO: `/v3/projects/861f3394e7e24b7697f45280f38bb48c` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:41.097852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=861f3394e7e24b7697f45280f38bb48c)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:41.099814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:41.257600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '861f3394e7e24b7697f45280f38bb48c', 'request_id': 'req-72301133-6d05-4906-9e3e-b7d5c2c4ed02'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:41.387497 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '861f3394e7e24b7697f45280f38bb48c', 'request_id': 'req-72301133-6d05-4906-9e3e-b7d5c2c4ed02'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:41.395111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '861f3394e7e24b7697f45280f38bb48c', 'request_id': 'req-72301133-6d05-4906-9e3e-b7d5c2c4ed02'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:41.405980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] The token cache is being invalidate because project 861f3394e7e24b7697f45280f38bb48c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:41.406282 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:41.406971 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:41.408550 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1358/2672] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:41 2022] DELETE /identity/v3/projects/861f3394e7e24b7697f45280f38bb48c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:43.543409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02f76f-66c1-4360-91f9-9f6aa72abca0 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:43.543930 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02f76f-66c1-4360-91f9-9f6aa72abca0 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:43.544048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02f76f-66c1-4360-91f9-9f6aa72abca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:43.552482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:43.564306 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-ac02f76f-66c1-4360-91f9-9f6aa72abca0 None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:43.565551 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-ac02f76f-66c1-4360-91f9-9f6aa72abca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:43.623782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1315/2673] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 15:57:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:43.638949 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:43.640449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:43.709780 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72301133-6d05-4906-9e3e-b7d5c2c4ed02 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:43.714852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5273b9f-82b8-44b6-b777-a4420247410a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:43.715456 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5273b9f-82b8-44b6-b777-a4420247410a service swift] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:43.715744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5273b9f-82b8-44b6-b777-a4420247410a service swift] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:43.716061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5273b9f-82b8-44b6-b777-a4420247410a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:43.724258 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5273b9f-82b8-44b6-b777-a4420247410a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:43.725878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5273b9f-82b8-44b6-b777-a4420247410a service swift] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:43.734281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1359/2674] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:57:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:57:53.667117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=23 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:53.730903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:05.865299 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c714f2b-cb22-489e-a43a-143236c84540 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:05.866043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c714f2b-cb22-489e-a43a-143236c84540 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:05.866276 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c714f2b-cb22-489e-a43a-143236c84540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:05.925174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-8c714f2b-cb22-489e-a43a-143236c84540 None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:05.927447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8c714f2b-cb22-489e-a43a-143236c84540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:05.967976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1316/2675] 10.222.0.98 () {58 vars in 967 bytes} [Fri Aug 26 15:58:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:05.993954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5273b9f-82b8-44b6-b777-a4420247410a service swift] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:06.050884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5273b9f-82b8-44b6-b777-a4420247410a service swift] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:06.058085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69772dc2-7bf8-44f8-a556-1dd7664152d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:06.059083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69772dc2-7bf8-44f8-a556-1dd7664152d9 service swift] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:06.059754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69772dc2-7bf8-44f8-a556-1dd7664152d9 service swift] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:06.060515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69772dc2-7bf8-44f8-a556-1dd7664152d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:06.070032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69772dc2-7bf8-44f8-a556-1dd7664152d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:06.071875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69772dc2-7bf8-44f8-a556-1dd7664152d9 service swift] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:06.081378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1360/2676] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 15:58:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:58:08.724725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c714f2b-cb22-489e-a43a-143236c84540 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:08.727257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8c714f2b-cb22-489e-a43a-143236c84540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:08.819658 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c714f2b-cb22-489e-a43a-143236c84540 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:08.823654 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e22c5-3c1c-4dec-bded-d47c978787ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:08.824331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e22c5-3c1c-4dec-bded-d47c978787ab service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:08.824615 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e22c5-3c1c-4dec-bded-d47c978787ab service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:08.824892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e22c5-3c1c-4dec-bded-d47c978787ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:08.827211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-828e22c5-3c1c-4dec-bded-d47c978787ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:08.927618 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e22c5-3c1c-4dec-bded-d47c978787ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:08.930314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e22c5-3c1c-4dec-bded-d47c978787ab service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:08.947801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1317/2677] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:16.077496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=20 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:17.555556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafffc51-9219-4bd1-bff4-0045f50ecfb4 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:17.556257 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafffc51-9219-4bd1-bff4-0045f50ecfb4 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:17.556568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafffc51-9219-4bd1-bff4-0045f50ecfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:17.584446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-aafffc51-9219-4bd1-bff4-0045f50ecfb4 None None] MFA Rules not processed for user `436adf49f3e24e03971017ea6dd33ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:17.586477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-aafffc51-9219-4bd1-bff4-0045f50ecfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:17.594151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1361/2678] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:58:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:18.651003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e22c5-3c1c-4dec-bded-d47c978787ab service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:18.652541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-828e22c5-3c1c-4dec-bded-d47c978787ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:18.767605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:18.787400 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e22c5-3c1c-4dec-bded-d47c978787ab service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:18.796576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64869c6a-6463-4328-a208-6a81d4fff52e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:18.797609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64869c6a-6463-4328-a208-6a81d4fff52e service placement] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:18.798103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64869c6a-6463-4328-a208-6a81d4fff52e service placement] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:18.799229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64869c6a-6463-4328-a208-6a81d4fff52e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:18.802356 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-64869c6a-6463-4328-a208-6a81d4fff52e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:18.824665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64869c6a-6463-4328-a208-6a81d4fff52e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:18.828042 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64869c6a-6463-4328-a208-6a81d4fff52e service placement] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:18.846333 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1318/2679] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:58:21.282997 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafffc51-9219-4bd1-bff4-0045f50ecfb4 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:21.284961 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-aafffc51-9219-4bd1-bff4-0045f50ecfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:21.363249 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafffc51-9219-4bd1-bff4-0045f50ecfb4 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:21.367348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027515d8-8d21-427e-a772-4352efed63fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:21.368093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027515d8-8d21-427e-a772-4352efed63fc service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:21.368482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027515d8-8d21-427e-a772-4352efed63fc service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:21.368831 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027515d8-8d21-427e-a772-4352efed63fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:21.370757 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-027515d8-8d21-427e-a772-4352efed63fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:21.528093 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027515d8-8d21-427e-a772-4352efed63fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:21.531664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027515d8-8d21-427e-a772-4352efed63fc service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:21.568240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1362/2680] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:27.230570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64869c6a-6463-4328-a208-6a81d4fff52e service placement] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:27.232511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-64869c6a-6463-4328-a208-6a81d4fff52e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:27.252021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64869c6a-6463-4328-a208-6a81d4fff52e service placement] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:27.256907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea982dcd-d6ca-4de5-8a50-88a754963721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:27.257285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea982dcd-d6ca-4de5-8a50-88a754963721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:27.257564 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea982dcd-d6ca-4de5-8a50-88a754963721 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:27.257838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea982dcd-d6ca-4de5-8a50-88a754963721 admin admin] PATH_INFO: `/v3/users/9348310fd06b473786d1c6be4f20c481` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:27.259026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea982dcd-d6ca-4de5-8a50-88a754963721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9348310fd06b473786d1c6be4f20c481)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:27.260989 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea982dcd-d6ca-4de5-8a50-88a754963721 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:27.474841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-ea982dcd-d6ca-4de5-8a50-88a754963721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9348310fd06b473786d1c6be4f20c481'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:27.475417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-ea982dcd-d6ca-4de5-8a50-88a754963721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9348310fd06b473786d1c6be4f20c481'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:27.489509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-ea982dcd-d6ca-4de5-8a50-88a754963721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9348310fd06b473786d1c6be4f20c481'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:27.571605 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-ea982dcd-d6ca-4de5-8a50-88a754963721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9348310fd06b473786d1c6be4f20c481'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:27.582057 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-ea982dcd-d6ca-4de5-8a50-88a754963721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9348310fd06b473786d1c6be4f20c481'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:27.590815 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1319/2681] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:58:27 2022] DELETE /identity/v3/users/9348310fd06b473786d1c6be4f20c481 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:27.915075 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027515d8-8d21-427e-a772-4352efed63fc service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:27.933290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027515d8-8d21-427e-a772-4352efed63fc service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:28.028486 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:28.029201 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:28.029481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:28.029754 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] PATH_INFO: `/v3/projects/f40d6476c750438c9c09107c7757fa95` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:28.031011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f40d6476c750438c9c09107c7757fa95)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:28.033762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:28.055226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:28.188023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f40d6476c750438c9c09107c7757fa95', 'request_id': 'req-38a2d1ea-9463-4823-b2f2-98aff8f84b76'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:28.268853 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f40d6476c750438c9c09107c7757fa95', 'request_id': 'req-38a2d1ea-9463-4823-b2f2-98aff8f84b76'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:28.278689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f40d6476c750438c9c09107c7757fa95', 'request_id': 'req-38a2d1ea-9463-4823-b2f2-98aff8f84b76'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:28.288477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] The token cache is being invalidate because project f40d6476c750438c9c09107c7757fa95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:58:28.288808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:28.289548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-38a2d1ea-9463-4823-b2f2-98aff8f84b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:28.291360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1363/2682] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:58:27 2022] DELETE /identity/v3/projects/f40d6476c750438c9c09107c7757fa95 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:58:28.296936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01598202-2180-4679-b6f5-f32540a045f5 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:28.297546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01598202-2180-4679-b6f5-f32540a045f5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:28.297546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01598202-2180-4679-b6f5-f32540a045f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:28.363176 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-01598202-2180-4679-b6f5-f32540a045f5 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:28.364479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-01598202-2180-4679-b6f5-f32540a045f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:28.429952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1320/2683] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:28.438011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd3733-06fb-4709-a84c-485f3b01df15 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:28.438387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd3733-06fb-4709-a84c-485f3b01df15 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:28.439283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd3733-06fb-4709-a84c-485f3b01df15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:28.460545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9efd3733-06fb-4709-a84c-485f3b01df15 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:28.461790 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9efd3733-06fb-4709-a84c-485f3b01df15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:28.472838 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1364/2684] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:58:28.477200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01598202-2180-4679-b6f5-f32540a045f5 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:28.487591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01598202-2180-4679-b6f5-f32540a045f5 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:28.492661 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cd09f-7007-40d5-b737-266815373c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:28.493292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cd09f-7007-40d5-b737-266815373c30 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:28.493576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cd09f-7007-40d5-b737-266815373c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:28.493854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cd09f-7007-40d5-b737-266815373c30 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:28.494521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cd09f-7007-40d5-b737-266815373c30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:58:28.494829 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cd09f-7007-40d5-b737-266815373c30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:28.496505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cd09f-7007-40d5-b737-266815373c30 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:28.510398 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1321/2685] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:58:28.515113 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd3733-06fb-4709-a84c-485f3b01df15 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:28.525827 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd3733-06fb-4709-a84c-485f3b01df15 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:28.531016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347577b-d310-4f4e-9091-9cae1742270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:28.531546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347577b-d310-4f4e-9091-9cae1742270c admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:28.531830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347577b-d310-4f4e-9091-9cae1742270c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:28.532181 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347577b-d310-4f4e-9091-9cae1742270c admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:28.533465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347577b-d310-4f4e-9091-9cae1742270c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:28.535401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347577b-d310-4f4e-9091-9cae1742270c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:28.641150 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1365/2686] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:28 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:28.646555 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cd09f-7007-40d5-b737-266815373c30 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:28.658004 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cd09f-7007-40d5-b737-266815373c30 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:28.667445 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:28.728237 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db5f28-1717-49c8-989e-98c0f2c39ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:28.728953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db5f28-1717-49c8-989e-98c0f2c39ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:28.729905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db5f28-1717-49c8-989e-98c0f2c39ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:28.730331 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db5f28-1717-49c8-989e-98c0f2c39ea7 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:28.731007 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db5f28-1717-49c8-989e-98c0f2c39ea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:28.733309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db5f28-1717-49c8-989e-98c0f2c39ea7 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:28.883148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1322/2687] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:28 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:28.888341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347577b-d310-4f4e-9091-9cae1742270c admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:28.903815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347577b-d310-4f4e-9091-9cae1742270c admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:28.912978 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bbfcf-211e-44dc-9ca3-16a90b32dc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:28.914977 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bbfcf-211e-44dc-9ca3-16a90b32dc02 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:28.915467 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bbfcf-211e-44dc-9ca3-16a90b32dc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:28.916101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bbfcf-211e-44dc-9ca3-16a90b32dc02 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:28.917143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bbfcf-211e-44dc-9ca3-16a90b32dc02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:28.920317 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bbfcf-211e-44dc-9ca3-16a90b32dc02 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:28.939856 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1366/2688] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:28.944601 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db5f28-1717-49c8-989e-98c0f2c39ea7 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:28.957752 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db5f28-1717-49c8-989e-98c0f2c39ea7 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:28.963667 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6ba31-d527-4a97-b0e9-9f279a1fe34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:28.964413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ba31-d527-4a97-b0e9-9f279a1fe34e admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:28.964744 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ba31-d527-4a97-b0e9-9f279a1fe34e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:28.965069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ba31-d527-4a97-b0e9-9f279a1fe34e admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:28.965801 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6ba31-d527-4a97-b0e9-9f279a1fe34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:28.967548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6ba31-d527-4a97-b0e9-9f279a1fe34e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:28.981583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1323/2689] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:58:28.986672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bbfcf-211e-44dc-9ca3-16a90b32dc02 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:28.998288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bbfcf-211e-44dc-9ca3-16a90b32dc02 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:29.003862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35392026-21cb-4422-9eca-1b273bbe5bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:29.004730 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35392026-21cb-4422-9eca-1b273bbe5bde admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.005135 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35392026-21cb-4422-9eca-1b273bbe5bde admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.005461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35392026-21cb-4422-9eca-1b273bbe5bde admin admin] PATH_INFO: `/v3/projects/77593ccc79d94240bd1a07c683844214/users/073949de3e364756be8f243a2cb8450d/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.023751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35392026-21cb-4422-9eca-1b273bbe5bde admin admin] RBAC: Authorizing `identity:create_grant(project_id=77593ccc79d94240bd1a07c683844214, user_id=073949de3e364756be8f243a2cb8450d, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:29.026464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35392026-21cb-4422-9eca-1b273bbe5bde admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:29.138105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1367/2690] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:28 2022] PUT /identity/v3/projects/77593ccc79d94240bd1a07c683844214/users/073949de3e364756be8f243a2cb8450d/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:29.147529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3405d-97e4-4d1e-aa57-b1637e270ea4 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.148067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3405d-97e4-4d1e-aa57-b1637e270ea4 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.148351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3405d-97e4-4d1e-aa57-b1637e270ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.182529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-47c3405d-97e4-4d1e-aa57-b1637e270ea4 None None] MFA Rules not processed for user `073949de3e364756be8f243a2cb8450d`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:29.184175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-47c3405d-97e4-4d1e-aa57-b1637e270ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:29.290616 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1324/2691] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:58:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:29.296636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35392026-21cb-4422-9eca-1b273bbe5bde admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:29.403708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35392026-21cb-4422-9eca-1b273bbe5bde admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:29.409304 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71bd32-293f-4850-8c82-555737657dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:29.410113 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71bd32-293f-4850-8c82-555737657dfc admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.410521 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71bd32-293f-4850-8c82-555737657dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.410918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71bd32-293f-4850-8c82-555737657dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.411678 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71bd32-293f-4850-8c82-555737657dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:29.413896 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71bd32-293f-4850-8c82-555737657dfc admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:29.512395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1368/2692] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:29 2022] POST /identity/v3/projects => generated 373 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:29.518010 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3405d-97e4-4d1e-aa57-b1637e270ea4 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:29.537292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3405d-97e4-4d1e-aa57-b1637e270ea4 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:29.622558 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9950ae7-773a-40c1-ab24-d02a4688ec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:29.623199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9950ae7-773a-40c1-ab24-d02a4688ec51 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.623497 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9950ae7-773a-40c1-ab24-d02a4688ec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.623783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9950ae7-773a-40c1-ab24-d02a4688ec51 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.624469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9950ae7-773a-40c1-ab24-d02a4688ec51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:29.627049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9950ae7-773a-40c1-ab24-d02a4688ec51 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:29.763292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1325/2693] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:29 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:29.769300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71bd32-293f-4850-8c82-555737657dfc admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:29.781395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71bd32-293f-4850-8c82-555737657dfc admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:29.787785 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c1156-44b1-4c08-8203-60621b2d5964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:29.788583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c1156-44b1-4c08-8203-60621b2d5964 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.788910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c1156-44b1-4c08-8203-60621b2d5964 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.789233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c1156-44b1-4c08-8203-60621b2d5964 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.790122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c1156-44b1-4c08-8203-60621b2d5964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:29.792032 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c1156-44b1-4c08-8203-60621b2d5964 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:29.804211 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1369/2694] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:29.810151 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9950ae7-773a-40c1-ab24-d02a4688ec51 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:29.821779 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9950ae7-773a-40c1-ab24-d02a4688ec51 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:29.827189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e859fc9-57e9-4e85-9ae6-c58b503a3ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:29.827907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e859fc9-57e9-4e85-9ae6-c58b503a3ead admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.828200 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e859fc9-57e9-4e85-9ae6-c58b503a3ead admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.828586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e859fc9-57e9-4e85-9ae6-c58b503a3ead admin admin] PATH_INFO: `/v3/projects/0a793477b8294de891fc49abfefd470f/users/bb3f999b5a9d488cb777b52dbf63c856/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.832788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:29.847972 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e859fc9-57e9-4e85-9ae6-c58b503a3ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a793477b8294de891fc49abfefd470f, user_id=bb3f999b5a9d488cb777b52dbf63c856, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:29.850694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e859fc9-57e9-4e85-9ae6-c58b503a3ead admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:29.991594 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1326/2695] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:29 2022] PUT /identity/v3/projects/0a793477b8294de891fc49abfefd470f/users/bb3f999b5a9d488cb777b52dbf63c856/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:58:29.996090 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c1156-44b1-4c08-8203-60621b2d5964 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:30.007623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c1156-44b1-4c08-8203-60621b2d5964 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:30.028992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:30.092175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90c350-ae5f-4ebc-97dc-6a15c2407a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:30.092809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c350-ae5f-4ebc-97dc-6a15c2407a8d admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:30.093099 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c350-ae5f-4ebc-97dc-6a15c2407a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:30.093557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c350-ae5f-4ebc-97dc-6a15c2407a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:30.094262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90c350-ae5f-4ebc-97dc-6a15c2407a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:30.096474 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90c350-ae5f-4ebc-97dc-6a15c2407a8d admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:30.107742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1370/2696] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:30.114158 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e859fc9-57e9-4e85-9ae6-c58b503a3ead admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:30.125790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e859fc9-57e9-4e85-9ae6-c58b503a3ead admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:30.131850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f968951-e774-4f80-a410-f1664d040180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:30.132848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968951-e774-4f80-a410-f1664d040180 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:30.133505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968951-e774-4f80-a410-f1664d040180 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:30.134459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968951-e774-4f80-a410-f1664d040180 admin admin] PATH_INFO: `/v3/projects/0a793477b8294de891fc49abfefd470f/users/bb3f999b5a9d488cb777b52dbf63c856/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:30.137423 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f968951-e774-4f80-a410-f1664d040180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a793477b8294de891fc49abfefd470f, user_id=bb3f999b5a9d488cb777b52dbf63c856, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:30.141561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f968951-e774-4f80-a410-f1664d040180 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:30.157173 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1327/2697] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:30 2022] PUT /identity/v3/projects/0a793477b8294de891fc49abfefd470f/users/bb3f999b5a9d488cb777b52dbf63c856/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:30.168121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb8079-4254-4c88-b56f-6d5c50bb2040 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:30.168936 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb8079-4254-4c88-b56f-6d5c50bb2040 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:30.169556 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb8079-4254-4c88-b56f-6d5c50bb2040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:30.202089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-a6cb8079-4254-4c88-b56f-6d5c50bb2040 None None] MFA Rules not processed for user `bb3f999b5a9d488cb777b52dbf63c856`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:30.204987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a6cb8079-4254-4c88-b56f-6d5c50bb2040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:30.292675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1371/2698] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:58:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:58:30.307071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f968951-e774-4f80-a410-f1664d040180 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:30.309391 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-2f968951-e774-4f80-a410-f1664d040180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:30.419998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f968951-e774-4f80-a410-f1664d040180 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:30.427390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226baebe-dd40-4518-bc5d-f7566abb8931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:30.429281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226baebe-dd40-4518-bc5d-f7566abb8931 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:30.429764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226baebe-dd40-4518-bc5d-f7566abb8931 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:30.430227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226baebe-dd40-4518-bc5d-f7566abb8931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:30.446923 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226baebe-dd40-4518-bc5d-f7566abb8931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:30.449462 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226baebe-dd40-4518-bc5d-f7566abb8931 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:30.462231 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1328/2699] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:30.749408 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb8079-4254-4c88-b56f-6d5c50bb2040 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:30.760170 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb8079-4254-4c88-b56f-6d5c50bb2040 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:30.763946 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047f53a-c4d5-4a57-ad7d-750a60e7cef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:30.764573 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047f53a-c4d5-4a57-ad7d-750a60e7cef0 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:30.765028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047f53a-c4d5-4a57-ad7d-750a60e7cef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:30.765307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047f53a-c4d5-4a57-ad7d-750a60e7cef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:30.841955 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047f53a-c4d5-4a57-ad7d-750a60e7cef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:30.845415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047f53a-c4d5-4a57-ad7d-750a60e7cef0 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:30.862665 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1372/2700] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:58:39.382352 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226baebe-dd40-4518-bc5d-f7566abb8931 service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:39.383964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-226baebe-dd40-4518-bc5d-f7566abb8931 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:39.484160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226baebe-dd40-4518-bc5d-f7566abb8931 service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:39.488457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20181a53-7698-40bc-8623-61c0eb841399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:39.489109 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20181a53-7698-40bc-8623-61c0eb841399 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:39.489387 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20181a53-7698-40bc-8623-61c0eb841399 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:39.489659 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20181a53-7698-40bc-8623-61c0eb841399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:39.491432 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-20181a53-7698-40bc-8623-61c0eb841399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:39.543807 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:39.563664 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20181a53-7698-40bc-8623-61c0eb841399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:39.566365 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20181a53-7698-40bc-8623-61c0eb841399 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:39.582589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1329/2701] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:58:40.854561 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=84 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:49.738119 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=7 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:58.335773 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0da14-fb34-4440-976b-4ca9be977db2 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:58.336811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0da14-fb34-4440-976b-4ca9be977db2 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:58.336811 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0da14-fb34-4440-976b-4ca9be977db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:58.360599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-87d0da14-fb34-4440-976b-4ca9be977db2 None None] MFA Rules not processed for user `91966686fa7f4c5bb326cdb1e2abc1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:58.362559 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-87d0da14-fb34-4440-976b-4ca9be977db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:58.447004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1373/2702] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:58:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:08.434938 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=25 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:22.293269 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20181a53-7698-40bc-8623-61c0eb841399 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:22.384926 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20181a53-7698-40bc-8623-61c0eb841399 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:22.391118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:22.391758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:22.392084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:22.392461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] PATH_INFO: `/v3/users/073949de3e364756be8f243a2cb8450d` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:22.393520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=073949de3e364756be8f243a2cb8450d)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:22.395578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:22.566793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073949de3e364756be8f243a2cb8450d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:22.567550 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073949de3e364756be8f243a2cb8450d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:22.586296 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073949de3e364756be8f243a2cb8450d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:22.713686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073949de3e364756be8f243a2cb8450d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:22.723190 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073949de3e364756be8f243a2cb8450d'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:22.729945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1330/2703] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:59:22 2022] DELETE /identity/v3/users/073949de3e364756be8f243a2cb8450d => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:22.735239 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60b7d-454f-49e1-8f2c-cf7c5cabb3e2 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:22.735766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60b7d-454f-49e1-8f2c-cf7c5cabb3e2 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:22.735890 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60b7d-454f-49e1-8f2c-cf7c5cabb3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:22.755266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-2dd60b7d-454f-49e1-8f2c-cf7c5cabb3e2 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:22.756530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2dd60b7d-454f-49e1-8f2c-cf7c5cabb3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:22.822192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1374/2704] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:22.833332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:22.834851 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:22.897573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6f9c4-6bcd-4fda-b260-f07f59a3cf66 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:22.901413 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70554d8e-5f71-450d-be88-0c0345c52557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:22.902076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70554d8e-5f71-450d-be88-0c0345c52557 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:22.902417 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70554d8e-5f71-450d-be88-0c0345c52557 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:22.902697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70554d8e-5f71-450d-be88-0c0345c52557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:22.913443 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70554d8e-5f71-450d-be88-0c0345c52557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:22.914897 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70554d8e-5f71-450d-be88-0c0345c52557 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:22.925529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1331/2705] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:59:23.350784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd60b7d-454f-49e1-8f2c-cf7c5cabb3e2 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:23.352522 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2dd60b7d-454f-49e1-8f2c-cf7c5cabb3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:23.365615 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd60b7d-454f-49e1-8f2c-cf7c5cabb3e2 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:23.371077 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:23.371822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:23.372184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:23.372960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] PATH_INFO: `/v3/projects/77593ccc79d94240bd1a07c683844214` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:23.374365 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=77593ccc79d94240bd1a07c683844214)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:23.376281 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:23.487155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77593ccc79d94240bd1a07c683844214', 'request_id': 'req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:23.558923 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77593ccc79d94240bd1a07c683844214', 'request_id': 'req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:23.566846 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77593ccc79d94240bd1a07c683844214', 'request_id': 'req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:23.579701 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] The token cache is being invalidate because project 77593ccc79d94240bd1a07c683844214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:59:23.580143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:23.580867 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:23.582601 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1375/2706] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:59:23 2022] DELETE /identity/v3/projects/77593ccc79d94240bd1a07c683844214 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:59:23.588212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70554d8e-5f71-450d-be88-0c0345c52557 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:23.599259 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70554d8e-5f71-450d-be88-0c0345c52557 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:23.619180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:23.668938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d64574-3c27-4333-aca1-bf10d5ba945c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:23.670108 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d64574-3c27-4333-aca1-bf10d5ba945c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:23.670624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d64574-3c27-4333-aca1-bf10d5ba945c admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:23.671133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d64574-3c27-4333-aca1-bf10d5ba945c admin admin] PATH_INFO: `/v3/users/bb3f999b5a9d488cb777b52dbf63c856` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:23.673034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d64574-3c27-4333-aca1-bf10d5ba945c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb3f999b5a9d488cb777b52dbf63c856)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:23.677028 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d64574-3c27-4333-aca1-bf10d5ba945c admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:23.945530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-75d64574-3c27-4333-aca1-bf10d5ba945c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb3f999b5a9d488cb777b52dbf63c856'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:23.946308 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-75d64574-3c27-4333-aca1-bf10d5ba945c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb3f999b5a9d488cb777b52dbf63c856'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:23.966773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-75d64574-3c27-4333-aca1-bf10d5ba945c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb3f999b5a9d488cb777b52dbf63c856'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.045825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-75d64574-3c27-4333-aca1-bf10d5ba945c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb3f999b5a9d488cb777b52dbf63c856'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.058629 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-75d64574-3c27-4333-aca1-bf10d5ba945c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb3f999b5a9d488cb777b52dbf63c856'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.068185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1332/2707] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:59:23 2022] DELETE /identity/v3/users/bb3f999b5a9d488cb777b52dbf63c856 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:24.521434 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:24.523998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:24.542653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534f159-b40a-4ca7-9b7e-d4742d5a6b0b admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:24.626400 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:24.627091 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:24.627374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:24.627649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] PATH_INFO: `/v3/projects/0a793477b8294de891fc49abfefd470f` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:24.628932 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a793477b8294de891fc49abfefd470f)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:24.630887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:24.748836 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:24.767714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a793477b8294de891fc49abfefd470f', 'request_id': 'req-12748c3d-fa7b-44cc-ad28-48146dc350a4'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.844501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a793477b8294de891fc49abfefd470f', 'request_id': 'req-12748c3d-fa7b-44cc-ad28-48146dc350a4'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.852041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a793477b8294de891fc49abfefd470f', 'request_id': 'req-12748c3d-fa7b-44cc-ad28-48146dc350a4'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.864976 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] The token cache is being invalidate because project 0a793477b8294de891fc49abfefd470f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:59:24.865219 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.865894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-12748c3d-fa7b-44cc-ad28-48146dc350a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.867389 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1376/2708] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:59:24 2022] DELETE /identity/v3/projects/0a793477b8294de891fc49abfefd470f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:59:24.874762 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d805ff-1883-4edc-a356-9efcbf2ba96c None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:24.875307 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d805ff-1883-4edc-a356-9efcbf2ba96c None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:24.875430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d805ff-1883-4edc-a356-9efcbf2ba96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:24.897121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-44d805ff-1883-4edc-a356-9efcbf2ba96c None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:24.898374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-44d805ff-1883-4edc-a356-9efcbf2ba96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:24.966866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1333/2709] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:24.975326 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bb8da-0ae4-407e-9e55-019a9b7120a9 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:24.975597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bb8da-0ae4-407e-9e55-019a9b7120a9 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:24.975919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bb8da-0ae4-407e-9e55-019a9b7120a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:24.997625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-023bb8da-0ae4-407e-9e55-019a9b7120a9 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:24.998935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-023bb8da-0ae4-407e-9e55-019a9b7120a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:25.007226 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1377/2710] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:25.012017 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d805ff-1883-4edc-a356-9efcbf2ba96c None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:25.023067 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d805ff-1883-4edc-a356-9efcbf2ba96c None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:25.028055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8539c4-f712-47b7-a2ee-443c10d8f555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:25.029222 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8539c4-f712-47b7-a2ee-443c10d8f555 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:25.029509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8539c4-f712-47b7-a2ee-443c10d8f555 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:25.029788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8539c4-f712-47b7-a2ee-443c10d8f555 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:25.030464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8539c4-f712-47b7-a2ee-443c10d8f555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:59:25.030775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8539c4-f712-47b7-a2ee-443c10d8f555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:25.032428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8539c4-f712-47b7-a2ee-443c10d8f555 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:25.045121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1334/2711] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:59:25.049919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bb8da-0ae4-407e-9e55-019a9b7120a9 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:25.060344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bb8da-0ae4-407e-9e55-019a9b7120a9 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:25.066003 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a2bf-2119-469c-a8fc-b02d5b7548de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:25.066791 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a2bf-2119-469c-a8fc-b02d5b7548de admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:25.067168 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a2bf-2119-469c-a8fc-b02d5b7548de admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:25.067541 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a2bf-2119-469c-a8fc-b02d5b7548de admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:25.068376 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662a2bf-2119-469c-a8fc-b02d5b7548de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:25.070524 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662a2bf-2119-469c-a8fc-b02d5b7548de admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:25.166417 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1378/2712] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:59:25 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:59:25.171019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8539c4-f712-47b7-a2ee-443c10d8f555 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:25.186382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8539c4-f712-47b7-a2ee-443c10d8f555 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:25.221094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:25.269702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe8d6d-b20e-4aea-8bec-daf970fc0fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:25.270300 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe8d6d-b20e-4aea-8bec-daf970fc0fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:25.270701 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe8d6d-b20e-4aea-8bec-daf970fc0fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:25.271189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe8d6d-b20e-4aea-8bec-daf970fc0fa3 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:25.271945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe8d6d-b20e-4aea-8bec-daf970fc0fa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:25.274422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe8d6d-b20e-4aea-8bec-daf970fc0fa3 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:25.812338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1335/2713] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:59:25 2022] POST /identity/v3/users => generated 289 bytes in 642 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:59:25.818851 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a2bf-2119-469c-a8fc-b02d5b7548de admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:25.838610 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a2bf-2119-469c-a8fc-b02d5b7548de admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:25.846963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e603f05-ba1e-499a-b782-253cccedc755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:25.848021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e603f05-ba1e-499a-b782-253cccedc755 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:25.848516 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e603f05-ba1e-499a-b782-253cccedc755 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:25.849012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e603f05-ba1e-499a-b782-253cccedc755 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:25.850260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e603f05-ba1e-499a-b782-253cccedc755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:25.853664 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e603f05-ba1e-499a-b782-253cccedc755 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:25.873944 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1379/2714] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:59:25.879375 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe8d6d-b20e-4aea-8bec-daf970fc0fa3 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:25.891676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe8d6d-b20e-4aea-8bec-daf970fc0fa3 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:25.897226 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c7a18-7f39-4ee1-900e-dca559481ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:25.897944 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7a18-7f39-4ee1-900e-dca559481ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:25.898277 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7a18-7f39-4ee1-900e-dca559481ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:25.898679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7a18-7f39-4ee1-900e-dca559481ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:25.899466 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c7a18-7f39-4ee1-900e-dca559481ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:25.901385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c7a18-7f39-4ee1-900e-dca559481ba3 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:25.915052 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1336/2715] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:59:25.920667 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e603f05-ba1e-499a-b782-253cccedc755 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:25.933283 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e603f05-ba1e-499a-b782-253cccedc755 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:25.951706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85911aa-2f5f-48b9-a947-92d750f96607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:25.953071 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85911aa-2f5f-48b9-a947-92d750f96607 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:25.953530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85911aa-2f5f-48b9-a947-92d750f96607 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:25.954035 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85911aa-2f5f-48b9-a947-92d750f96607 admin admin] PATH_INFO: `/v3/projects/695495d130b44d6a8bb0d97ea0776c35/users/05b112b76af94395abe8f08b58b39ec8/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:25.986495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85911aa-2f5f-48b9-a947-92d750f96607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=695495d130b44d6a8bb0d97ea0776c35, user_id=05b112b76af94395abe8f08b58b39ec8, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:25.989517 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85911aa-2f5f-48b9-a947-92d750f96607 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:26.064534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1380/2716] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:59:25 2022] PUT /identity/v3/projects/695495d130b44d6a8bb0d97ea0776c35/users/05b112b76af94395abe8f08b58b39ec8/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:59:26.080220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d637bf-06f4-4ed2-b776-e73d60b73a46 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:26.080740 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d637bf-06f4-4ed2-b776-e73d60b73a46 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:26.081346 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d637bf-06f4-4ed2-b776-e73d60b73a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:26.128490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-75d637bf-06f4-4ed2-b776-e73d60b73a46 None None] MFA Rules not processed for user `05b112b76af94395abe8f08b58b39ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:26.129645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-75d637bf-06f4-4ed2-b776-e73d60b73a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:26.205663 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1337/2717] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:59:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:26.221037 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85911aa-2f5f-48b9-a947-92d750f96607 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:26.222837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f85911aa-2f5f-48b9-a947-92d750f96607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:26.290481 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85911aa-2f5f-48b9-a947-92d750f96607 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:26.307143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc67f77-4019-4d04-bd6b-049eaf918621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:26.308234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc67f77-4019-4d04-bd6b-049eaf918621 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:26.308762 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc67f77-4019-4d04-bd6b-049eaf918621 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:26.309263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc67f77-4019-4d04-bd6b-049eaf918621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:26.327300 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc67f77-4019-4d04-bd6b-049eaf918621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:26.330339 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc67f77-4019-4d04-bd6b-049eaf918621 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:26.348809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1381/2718] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:59:36.202102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:36.340002 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:52.245355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d637bf-06f4-4ed2-b776-e73d60b73a46 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:52.357905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d637bf-06f4-4ed2-b776-e73d60b73a46 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:52.364948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:52.365927 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.366370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.366803 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] PATH_INFO: `/v3/users/05b112b76af94395abe8f08b58b39ec8` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.368755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b112b76af94395abe8f08b58b39ec8)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:52.372715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:52.550854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b112b76af94395abe8f08b58b39ec8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:52.552279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b112b76af94395abe8f08b58b39ec8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:52.574568 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b112b76af94395abe8f08b58b39ec8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:52.638166 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b112b76af94395abe8f08b58b39ec8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:52.647322 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b112b76af94395abe8f08b58b39ec8'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:52.654156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1338/2719] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 15:59:52 2022] DELETE /identity/v3/users/05b112b76af94395abe8f08b58b39ec8 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:52.659186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94707d24-80d6-47fc-8786-b0e55b02ed82 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.659710 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94707d24-80d6-47fc-8786-b0e55b02ed82 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.659897 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94707d24-80d6-47fc-8786-b0e55b02ed82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.682078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-94707d24-80d6-47fc-8786-b0e55b02ed82 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:52.684391 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-94707d24-80d6-47fc-8786-b0e55b02ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:52.751529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1382/2720] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:52.763009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:52.764620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:52.831265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8e8c7-885d-427b-bdb9-ad4a232e5f11 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:52.835510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d26f1-a753-444f-aa67-6716cabad49e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:52.836316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d26f1-a753-444f-aa67-6716cabad49e service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.836645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d26f1-a753-444f-aa67-6716cabad49e service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.836963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d26f1-a753-444f-aa67-6716cabad49e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.848506 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d26f1-a753-444f-aa67-6716cabad49e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:52.850031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d26f1-a753-444f-aa67-6716cabad49e service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:52.861397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1339/2721] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:59:53.395127 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94707d24-80d6-47fc-8786-b0e55b02ed82 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:53.397887 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-94707d24-80d6-47fc-8786-b0e55b02ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:53.423088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94707d24-80d6-47fc-8786-b0e55b02ed82 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:53.431696 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:53.432815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:53.434341 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:53.435088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] PATH_INFO: `/v3/projects/695495d130b44d6a8bb0d97ea0776c35` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:53.437329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=695495d130b44d6a8bb0d97ea0776c35)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:53.441302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:53.566637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695495d130b44d6a8bb0d97ea0776c35', 'request_id': 'req-4fd6b626-9b37-4e3d-807c-15be623baf7b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:53.643424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695495d130b44d6a8bb0d97ea0776c35', 'request_id': 'req-4fd6b626-9b37-4e3d-807c-15be623baf7b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:53.651183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695495d130b44d6a8bb0d97ea0776c35', 'request_id': 'req-4fd6b626-9b37-4e3d-807c-15be623baf7b'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:53.663108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] The token cache is being invalidate because project 695495d130b44d6a8bb0d97ea0776c35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:59:53.663788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:53.664431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-4fd6b626-9b37-4e3d-807c-15be623baf7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:53.667056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1383/2722] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 15:59:53 2022] DELETE /identity/v3/projects/695495d130b44d6a8bb0d97ea0776c35 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:53.672229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6865ca7-14d7-4fb0-9cef-d133088f92d6 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:53.672820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6865ca7-14d7-4fb0-9cef-d133088f92d6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:53.672820 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6865ca7-14d7-4fb0-9cef-d133088f92d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:53.694386 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-f6865ca7-14d7-4fb0-9cef-d133088f92d6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:53.695630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f6865ca7-14d7-4fb0-9cef-d133088f92d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:53.717808 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:53.763460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1340/2723] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:59:53.771904 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2b4de-96c2-41f2-9ff0-3a3c0d61ed8f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:53.772198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2b4de-96c2-41f2-9ff0-3a3c0d61ed8f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:53.772663 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2b4de-96c2-41f2-9ff0-3a3c0d61ed8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:53.795042 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-fab2b4de-96c2-41f2-9ff0-3a3c0d61ed8f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:53.796337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-fab2b4de-96c2-41f2-9ff0-3a3c0d61ed8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:53.804758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1384/2724] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:53.813971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6865ca7-14d7-4fb0-9cef-d133088f92d6 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:53.825763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6865ca7-14d7-4fb0-9cef-d133088f92d6 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:53.831152 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cce47-e43c-484d-a9fa-389c6705aa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:53.832548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cce47-e43c-484d-a9fa-389c6705aa25 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:53.833090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cce47-e43c-484d-a9fa-389c6705aa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:53.834125 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cce47-e43c-484d-a9fa-389c6705aa25 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:53.835694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cce47-e43c-484d-a9fa-389c6705aa25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:59:53.836280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cce47-e43c-484d-a9fa-389c6705aa25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:53.839495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cce47-e43c-484d-a9fa-389c6705aa25 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:53.859963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1341/2725] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 15:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:59:53.864412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2b4de-96c2-41f2-9ff0-3a3c0d61ed8f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:53.878016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2b4de-96c2-41f2-9ff0-3a3c0d61ed8f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:53.883246 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c74e3-d3d6-47fe-a2f5-c195be6a1893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:53.883894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c74e3-d3d6-47fe-a2f5-c195be6a1893 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:53.884210 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c74e3-d3d6-47fe-a2f5-c195be6a1893 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:53.884487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c74e3-d3d6-47fe-a2f5-c195be6a1893 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:53.885123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c74e3-d3d6-47fe-a2f5-c195be6a1893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:53.887677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c74e3-d3d6-47fe-a2f5-c195be6a1893 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:53.976054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1385/2726] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 15:59:53 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:59:53.980985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cce47-e43c-484d-a9fa-389c6705aa25 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:53.991481 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cce47-e43c-484d-a9fa-389c6705aa25 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:54.068848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc951b-da30-4502-91aa-328cb5d3fc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:54.069476 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc951b-da30-4502-91aa-328cb5d3fc4e admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:54.069770 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc951b-da30-4502-91aa-328cb5d3fc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:54.070048 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc951b-da30-4502-91aa-328cb5d3fc4e admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:54.070697 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc951b-da30-4502-91aa-328cb5d3fc4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:54.072998 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc951b-da30-4502-91aa-328cb5d3fc4e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:54.179579 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1342/2727] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 15:59:53 2022] POST /identity/v3/users => generated 284 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:59:54.185324 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c74e3-d3d6-47fe-a2f5-c195be6a1893 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:54.196102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c74e3-d3d6-47fe-a2f5-c195be6a1893 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:54.201850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270354a-1ffc-4779-b75c-2bdb3e0e189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:54.202510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270354a-1ffc-4779-b75c-2bdb3e0e189b admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:54.202784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270354a-1ffc-4779-b75c-2bdb3e0e189b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:54.203191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270354a-1ffc-4779-b75c-2bdb3e0e189b admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:54.203909 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270354a-1ffc-4779-b75c-2bdb3e0e189b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:54.205551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270354a-1ffc-4779-b75c-2bdb3e0e189b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:54.217107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1386/2728] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:59:54.221722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc951b-da30-4502-91aa-328cb5d3fc4e admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:54.233128 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc951b-da30-4502-91aa-328cb5d3fc4e admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:54.238006 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e4933-d8bd-4d38-a0d4-17398f83af2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:54.238621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e4933-d8bd-4d38-a0d4-17398f83af2c admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:54.238908 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e4933-d8bd-4d38-a0d4-17398f83af2c admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:54.239189 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e4933-d8bd-4d38-a0d4-17398f83af2c admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:54.239848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e4933-d8bd-4d38-a0d4-17398f83af2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:54.241514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e4933-d8bd-4d38-a0d4-17398f83af2c admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:54.253840 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1343/2729] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:59:54.258134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270354a-1ffc-4779-b75c-2bdb3e0e189b admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:54.268666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270354a-1ffc-4779-b75c-2bdb3e0e189b admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:54.273957 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c16114-bcee-4f14-90e0-e58ea17e18e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:54.274706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c16114-bcee-4f14-90e0-e58ea17e18e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:54.274990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c16114-bcee-4f14-90e0-e58ea17e18e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:54.275270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c16114-bcee-4f14-90e0-e58ea17e18e6 admin admin] PATH_INFO: `/v3/projects/547acc5b53c748e09932c2b0613a08fb/users/2e8d2c2dc17645e68f1084e2ddb5e1c2/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:54.291380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c16114-bcee-4f14-90e0-e58ea17e18e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=547acc5b53c748e09932c2b0613a08fb, user_id=2e8d2c2dc17645e68f1084e2ddb5e1c2, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:54.294612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c16114-bcee-4f14-90e0-e58ea17e18e6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:54.404603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1387/2730] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 15:59:54 2022] PUT /identity/v3/projects/547acc5b53c748e09932c2b0613a08fb/users/2e8d2c2dc17645e68f1084e2ddb5e1c2/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:54.415532 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b7bb1-afc6-45ea-b155-665a1d58bed1 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:54.415915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b7bb1-afc6-45ea-b155-665a1d58bed1 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:54.416249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b7bb1-afc6-45ea-b155-665a1d58bed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:54.446510 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-137b7bb1-afc6-45ea-b155-665a1d58bed1 None None] MFA Rules not processed for user `2e8d2c2dc17645e68f1084e2ddb5e1c2`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:54.448153 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-137b7bb1-afc6-45ea-b155-665a1d58bed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:54.521822 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1344/2731] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 15:59:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:54.534731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c16114-bcee-4f14-90e0-e58ea17e18e6 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:54.537491 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-78c16114-bcee-4f14-90e0-e58ea17e18e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:54.577428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:54.638958 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c16114-bcee-4f14-90e0-e58ea17e18e6 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:54.645652 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0f3ce-20f3-4f2c-9310-caeaece8e4cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cef193a54eff44f2b53716e3ec1a29cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:54.646716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0f3ce-20f3-4f2c-9310-caeaece8e4cb service glance] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:54.647284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0f3ce-20f3-4f2c-9310-caeaece8e4cb service glance] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:54.647786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0f3ce-20f3-4f2c-9310-caeaece8e4cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:54.663756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0f3ce-20f3-4f2c-9310-caeaece8e4cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:54.667143 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0f3ce-20f3-4f2c-9310-caeaece8e4cb service glance] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:54.683399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1388/2732] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:04.518670 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:04.711463 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=17 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:06.085406 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b7bb1-afc6-45ea-b155-665a1d58bed1 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:06.088093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-137b7bb1-afc6-45ea-b155-665a1d58bed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:06.161802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b7bb1-afc6-45ea-b155-665a1d58bed1 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:06.165646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2ef3e-0951-48d2-ae88-cb7c2548108b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:06.166267 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2ef3e-0951-48d2-ae88-cb7c2548108b service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:06.167164 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2ef3e-0951-48d2-ae88-cb7c2548108b service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:06.167440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2ef3e-0951-48d2-ae88-cb7c2548108b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:06.169392 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4ae2ef3e-0951-48d2-ae88-cb7c2548108b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:06.280725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2ef3e-0951-48d2-ae88-cb7c2548108b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:06.282130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2ef3e-0951-48d2-ae88-cb7c2548108b service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:06.305917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1345/2733] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:07.520212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0f3ce-20f3-4f2c-9310-caeaece8e4cb service glance] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:07.521822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-12b0f3ce-20f3-4f2c-9310-caeaece8e4cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:07.548669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0f3ce-20f3-4f2c-9310-caeaece8e4cb service glance] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:07.548669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc4bba-7a37-4bdd-ba97-d7bb47c6023d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:07.549100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc4bba-7a37-4bdd-ba97-d7bb47c6023d service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:07.549544 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc4bba-7a37-4bdd-ba97-d7bb47c6023d service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:07.550044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc4bba-7a37-4bdd-ba97-d7bb47c6023d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:07.553053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c1bc4bba-7a37-4bdd-ba97-d7bb47c6023d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:07.578801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc4bba-7a37-4bdd-ba97-d7bb47c6023d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:07.581772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc4bba-7a37-4bdd-ba97-d7bb47c6023d service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:07.601244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1389/2734] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:00:16.303201 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=63 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:17.199702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac74c4-4ab9-4510-ba6d-81a4dc1e263f None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:17.206009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac74c4-4ab9-4510-ba6d-81a4dc1e263f None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:17.206009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac74c4-4ab9-4510-ba6d-81a4dc1e263f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:17.235281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-28ac74c4-4ab9-4510-ba6d-81a4dc1e263f None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:17.236562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-28ac74c4-4ab9-4510-ba6d-81a4dc1e263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:17.306620 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1346/2735] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 16:00:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:00:17.464623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc4bba-7a37-4bdd-ba97-d7bb47c6023d service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:17.466182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c1bc4bba-7a37-4bdd-ba97-d7bb47c6023d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:17.540404 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc4bba-7a37-4bdd-ba97-d7bb47c6023d service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:17.545755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a2ada-bf47-4c92-a18c-80d03c9fbe9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:17.546396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a2ada-bf47-4c92-a18c-80d03c9fbe9e service swift] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:17.546669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a2ada-bf47-4c92-a18c-80d03c9fbe9e service swift] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:17.546943 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a2ada-bf47-4c92-a18c-80d03c9fbe9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:17.555466 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a2ada-bf47-4c92-a18c-80d03c9fbe9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:17.557756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a2ada-bf47-4c92-a18c-80d03c9fbe9e service swift] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:17.566329 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1390/2736] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 16:00:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 16:00:27.299062 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=29 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:27.562969 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=38 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:33.346339 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac74c4-4ab9-4510-ba6d-81a4dc1e263f None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:33.424464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac74c4-4ab9-4510-ba6d-81a4dc1e263f None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:33.431721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:33.432871 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:33.433433 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:33.439219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] PATH_INFO: `/v3/users/2e8d2c2dc17645e68f1084e2ddb5e1c2` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:33.439219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e8d2c2dc17645e68f1084e2ddb5e1c2)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:33.439524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:33.649136 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e8d2c2dc17645e68f1084e2ddb5e1c2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:33.650229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e8d2c2dc17645e68f1084e2ddb5e1c2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:33.668179 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e8d2c2dc17645e68f1084e2ddb5e1c2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:33.750103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e8d2c2dc17645e68f1084e2ddb5e1c2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:33.763249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e8d2c2dc17645e68f1084e2ddb5e1c2'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:33.776940 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1347/2737] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:00:33 2022] DELETE /identity/v3/users/2e8d2c2dc17645e68f1084e2ddb5e1c2 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:00:33.782960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd815c-4a27-4762-b1f7-f8b8d37459b1 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:33.783851 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd815c-4a27-4762-b1f7-f8b8d37459b1 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:33.783851 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd815c-4a27-4762-b1f7-f8b8d37459b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:33.812397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-67fd815c-4a27-4762-b1f7-f8b8d37459b1 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:33.813640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-67fd815c-4a27-4762-b1f7-f8b8d37459b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:33.912729 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1391/2738] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:00:33.926448 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:33.930917 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:34.010723 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871c0c6-81f5-4667-898a-f5af486ff367 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:34.014570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add6691-153d-40a5-815e-a91307a086d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:34.015191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add6691-153d-40a5-815e-a91307a086d5 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:34.015474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add6691-153d-40a5-815e-a91307a086d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:34.015750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add6691-153d-40a5-815e-a91307a086d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:34.028440 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add6691-153d-40a5-815e-a91307a086d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:34.030141 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add6691-153d-40a5-815e-a91307a086d5 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:34.042249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1348/2739] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:00:34.569500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd815c-4a27-4762-b1f7-f8b8d37459b1 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:34.572914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-67fd815c-4a27-4762-b1f7-f8b8d37459b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:34.622894 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd815c-4a27-4762-b1f7-f8b8d37459b1 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:34.630153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:34.630828 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:34.631116 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:34.631397 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] PATH_INFO: `/v3/projects/547acc5b53c748e09932c2b0613a08fb` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:34.632864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=547acc5b53c748e09932c2b0613a08fb)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:34.635403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:34.823438 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '547acc5b53c748e09932c2b0613a08fb', 'request_id': 'req-0741e1b5-7295-4eb7-b76c-a90d43844c22'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:34.933366 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '547acc5b53c748e09932c2b0613a08fb', 'request_id': 'req-0741e1b5-7295-4eb7-b76c-a90d43844c22'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:34.941158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '547acc5b53c748e09932c2b0613a08fb', 'request_id': 'req-0741e1b5-7295-4eb7-b76c-a90d43844c22'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:34.975706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] The token cache is being invalidate because project 547acc5b53c748e09932c2b0613a08fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:00:34.976121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:34.977189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-0741e1b5-7295-4eb7-b76c-a90d43844c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:34.979276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1392/2740] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:00:34 2022] DELETE /identity/v3/projects/547acc5b53c748e09932c2b0613a08fb => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:00:34.987511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03949fe-1357-40e5-b667-965121e98c28 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:34.988089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03949fe-1357-40e5-b667-965121e98c28 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:34.990216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03949fe-1357-40e5-b667-965121e98c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:35.030647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b03949fe-1357-40e5-b667-965121e98c28 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:35.032561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b03949fe-1357-40e5-b667-965121e98c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:35.068395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:35.129758 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1349/2741] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:35.138898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf13a9-af89-42c6-8fc1-729fb39b9ae3 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:35.139759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf13a9-af89-42c6-8fc1-729fb39b9ae3 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:35.139759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf13a9-af89-42c6-8fc1-729fb39b9ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:35.167918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-21bf13a9-af89-42c6-8fc1-729fb39b9ae3 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:35.168980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-21bf13a9-af89-42c6-8fc1-729fb39b9ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:35.180274 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1393/2742] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:00:35.186385 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03949fe-1357-40e5-b667-965121e98c28 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:35.200447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03949fe-1357-40e5-b667-965121e98c28 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:35.206160 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddbab9-521e-4af4-80d6-f0e12427d442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:35.206948 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddbab9-521e-4af4-80d6-f0e12427d442 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:35.207239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddbab9-521e-4af4-80d6-f0e12427d442 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:35.207524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddbab9-521e-4af4-80d6-f0e12427d442 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:35.208335 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ddbab9-521e-4af4-80d6-f0e12427d442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:00:35.208637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ddbab9-521e-4af4-80d6-f0e12427d442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:35.210475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ddbab9-521e-4af4-80d6-f0e12427d442 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:35.227469 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1350/2743] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 16:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:00:35.233993 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf13a9-af89-42c6-8fc1-729fb39b9ae3 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:35.247345 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf13a9-af89-42c6-8fc1-729fb39b9ae3 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:35.258321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cbba4-cd14-4aa0-b1b6-5d7880ea433e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:35.260152 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cbba4-cd14-4aa0-b1b6-5d7880ea433e admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:35.260669 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cbba4-cd14-4aa0-b1b6-5d7880ea433e admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:35.261088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cbba4-cd14-4aa0-b1b6-5d7880ea433e admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:35.262272 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cbba4-cd14-4aa0-b1b6-5d7880ea433e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:35.264782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cbba4-cd14-4aa0-b1b6-5d7880ea433e admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:35.428158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1394/2744] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 16:00:35 2022] POST /identity/v3/projects => generated 371 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:00:35.435517 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddbab9-521e-4af4-80d6-f0e12427d442 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:35.457682 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddbab9-521e-4af4-80d6-f0e12427d442 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:35.602334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddbb2f-8530-4a6b-9a1e-3b453f330e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:35.603092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddbb2f-8530-4a6b-9a1e-3b453f330e3c admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:35.603444 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddbb2f-8530-4a6b-9a1e-3b453f330e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:35.603782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddbb2f-8530-4a6b-9a1e-3b453f330e3c admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:35.606114 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddbb2f-8530-4a6b-9a1e-3b453f330e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:35.609983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddbb2f-8530-4a6b-9a1e-3b453f330e3c admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:35.723142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1351/2745] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 16:00:35 2022] POST /identity/v3/users => generated 286 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:00:35.728439 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cbba4-cd14-4aa0-b1b6-5d7880ea433e admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:35.746209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cbba4-cd14-4aa0-b1b6-5d7880ea433e admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:35.751844 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b185c6-0e58-45e2-bebd-7ba1d6b21f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:35.752563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b185c6-0e58-45e2-bebd-7ba1d6b21f0a admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:35.753016 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b185c6-0e58-45e2-bebd-7ba1d6b21f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:35.753416 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b185c6-0e58-45e2-bebd-7ba1d6b21f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:35.754083 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b185c6-0e58-45e2-bebd-7ba1d6b21f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:35.756056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b185c6-0e58-45e2-bebd-7ba1d6b21f0a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:35.772060 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1395/2746] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:00:35.778790 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddbb2f-8530-4a6b-9a1e-3b453f330e3c admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:35.789047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddbb2f-8530-4a6b-9a1e-3b453f330e3c admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:35.795455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21794c62-70d6-488e-b4b2-322f437c0e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:35.803408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21794c62-70d6-488e-b4b2-322f437c0e27 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:35.803408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21794c62-70d6-488e-b4b2-322f437c0e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:35.803408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21794c62-70d6-488e-b4b2-322f437c0e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:35.803408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21794c62-70d6-488e-b4b2-322f437c0e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:35.803408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21794c62-70d6-488e-b4b2-322f437c0e27 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:35.812728 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1352/2747] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:00:35.817833 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b185c6-0e58-45e2-bebd-7ba1d6b21f0a admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:35.831989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b185c6-0e58-45e2-bebd-7ba1d6b21f0a admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:35.838248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8476d9-44a4-456a-9081-8592e3b85e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:35.839256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8476d9-44a4-456a-9081-8592e3b85e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:35.839650 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8476d9-44a4-456a-9081-8592e3b85e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:35.840125 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8476d9-44a4-456a-9081-8592e3b85e2f admin admin] PATH_INFO: `/v3/projects/4d19ae3de8c54cbaa0317ea4950f0450/users/75f57356920a4d6ab3277bb0931349cc/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:35.861384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8476d9-44a4-456a-9081-8592e3b85e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d19ae3de8c54cbaa0317ea4950f0450, user_id=75f57356920a4d6ab3277bb0931349cc, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:35.864328 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8476d9-44a4-456a-9081-8592e3b85e2f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:35.933268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1396/2748] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:00:35 2022] PUT /identity/v3/projects/4d19ae3de8c54cbaa0317ea4950f0450/users/75f57356920a4d6ab3277bb0931349cc/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:00:35.951285 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7c069-7748-4843-bffd-38a9dcde5438 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:35.951713 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7c069-7748-4843-bffd-38a9dcde5438 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:35.952115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7c069-7748-4843-bffd-38a9dcde5438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:35.988464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-49b7c069-7748-4843-bffd-38a9dcde5438 None None] MFA Rules not processed for user `75f57356920a4d6ab3277bb0931349cc`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:35.990850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-49b7c069-7748-4843-bffd-38a9dcde5438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:36.082351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1353/2749] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:00:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:00:36.097302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8476d9-44a4-456a-9081-8592e3b85e2f admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:36.099368 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-1f8476d9-44a4-456a-9081-8592e3b85e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:36.122146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:36.181146 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8476d9-44a4-456a-9081-8592e3b85e2f admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:36.185656 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa63d91-6234-4515-81a5-469a5a6cf216 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cef193a54eff44f2b53716e3ec1a29cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:36.186577 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa63d91-6234-4515-81a5-469a5a6cf216 service glance] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:36.186863 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa63d91-6234-4515-81a5-469a5a6cf216 service glance] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:36.187174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa63d91-6234-4515-81a5-469a5a6cf216 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:36.198504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa63d91-6234-4515-81a5-469a5a6cf216 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:36.204473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa63d91-6234-4515-81a5-469a5a6cf216 service glance] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:36.210548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1397/2750] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:36.488370 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7c069-7748-4843-bffd-38a9dcde5438 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:36.489810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-49b7c069-7748-4843-bffd-38a9dcde5438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:36.510976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:36.557879 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7c069-7748-4843-bffd-38a9dcde5438 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:36.561570 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458a7ce-1ff0-4dab-9fd3-4b53f06a7ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:36.562185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458a7ce-1ff0-4dab-9fd3-4b53f06a7ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:36.562471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458a7ce-1ff0-4dab-9fd3-4b53f06a7ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:36.562751 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458a7ce-1ff0-4dab-9fd3-4b53f06a7ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:36.564624 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-2458a7ce-1ff0-4dab-9fd3-4b53f06a7ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:36.576211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458a7ce-1ff0-4dab-9fd3-4b53f06a7ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:36.579116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458a7ce-1ff0-4dab-9fd3-4b53f06a7ae7 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:36.588031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1354/2751] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:36.625540 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa63d91-6234-4515-81a5-469a5a6cf216 service glance] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:36.636503 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa63d91-6234-4515-81a5-469a5a6cf216 service glance] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:36.640464 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c1fb0-9472-41fc-b2a2-45cfa0c620e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:36.641250 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c1fb0-9472-41fc-b2a2-45cfa0c620e4 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:36.641571 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c1fb0-9472-41fc-b2a2-45cfa0c620e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:36.641860 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c1fb0-9472-41fc-b2a2-45cfa0c620e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:36.643616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f00c1fb0-9472-41fc-b2a2-45cfa0c620e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:36.657830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c1fb0-9472-41fc-b2a2-45cfa0c620e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:36.657830 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c1fb0-9472-41fc-b2a2-45cfa0c620e4 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:36.665847 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1398/2752] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:37.030001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e84a35-bd1a-4e4f-bf0d-0e37f5814841 None None] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.032096 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e84a35-bd1a-4e4f-bf0d-0e37f5814841 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.032383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e84a35-bd1a-4e4f-bf0d-0e37f5814841 None None] PATH_INFO: `/` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.038280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1355/2753] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 16:00:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 16:00:37.041169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a7372-10f3-43be-8ba1-023f68aae140 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.041169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a7372-10f3-43be-8ba1-023f68aae140 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.041169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a7372-10f3-43be-8ba1-023f68aae140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.093327 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-c10a7372-10f3-43be-8ba1-023f68aae140 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:37.094569 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c10a7372-10f3-43be-8ba1-023f68aae140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:37.219337 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1399/2754] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 16:00:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:00:37.235709 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e84a35-bd1a-4e4f-bf0d-0e37f5814841 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:37.238785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f4e84a35-bd1a-4e4f-bf0d-0e37f5814841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:37.256278 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e84a35-bd1a-4e4f-bf0d-0e37f5814841 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:37.262953 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543bc06-66a5-4d50-b78a-822b5a4564b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cef193a54eff44f2b53716e3ec1a29cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:37.267951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543bc06-66a5-4d50-b78a-822b5a4564b2 service glance] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.267951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543bc06-66a5-4d50-b78a-822b5a4564b2 service glance] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.267951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543bc06-66a5-4d50-b78a-822b5a4564b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.285044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543bc06-66a5-4d50-b78a-822b5a4564b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:37.287182 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543bc06-66a5-4d50-b78a-822b5a4564b2 service glance] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:37.301161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1356/2755] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:37.551196 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9a751-23a8-4046-aeab-0c779a984ef6 None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.551465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9a751-23a8-4046-aeab-0c779a984ef6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.551758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9a751-23a8-4046-aeab-0c779a984ef6 None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.553737 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1400/2756] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 16:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 16:00:37.559288 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b7abe-18df-4ab8-88a6-65d2e21d14c3 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.559582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b7abe-18df-4ab8-88a6-65d2e21d14c3 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.559910 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b7abe-18df-4ab8-88a6-65d2e21d14c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.607545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-364b7abe-18df-4ab8-88a6-65d2e21d14c3 None None] MFA Rules not processed for user `8c6cad18359143868276eb9d79a3fd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:37.608453 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-364b7abe-18df-4ab8-88a6-65d2e21d14c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:37.616220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1357/2757] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 16:00:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:37.630194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9a751-23a8-4046-aeab-0c779a984ef6 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:37.648009 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9a751-23a8-4046-aeab-0c779a984ef6 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:37.654741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586d6c1-5734-4409-80f7-ff9a839fb080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:37.656076 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586d6c1-5734-4409-80f7-ff9a839fb080 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.656429 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586d6c1-5734-4409-80f7-ff9a839fb080 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.656689 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586d6c1-5734-4409-80f7-ff9a839fb080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.665396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586d6c1-5734-4409-80f7-ff9a839fb080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:37.666857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586d6c1-5734-4409-80f7-ff9a839fb080 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:37.679672 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1401/2758] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:37.747329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b7abe-18df-4ab8-88a6-65d2e21d14c3 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:37.752665 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-364b7abe-18df-4ab8-88a6-65d2e21d14c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:37.771179 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b7abe-18df-4ab8-88a6-65d2e21d14c3 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:37.782811 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511158b-adf8-42ba-8269-55b6061d6406 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:37.783621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511158b-adf8-42ba-8269-55b6061d6406 service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.784044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511158b-adf8-42ba-8269-55b6061d6406 service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.784919 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511158b-adf8-42ba-8269-55b6061d6406 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.786847 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b0ce1-3b5d-464e-a0f4-1151d10cd41c None None] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.787211 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b0ce1-3b5d-464e-a0f4-1151d10cd41c None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.787487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b0ce1-3b5d-464e-a0f4-1151d10cd41c None None] PATH_INFO: `/` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.791962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1402/2759] 10.222.0.98 () {54 vars in 921 bytes} [Fri Aug 26 16:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 16:00:37.792149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7511158b-adf8-42ba-8269-55b6061d6406 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:37.798892 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07fa7b-0aad-4182-b7bf-3f7966b4a9b5 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.799138 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07fa7b-0aad-4182-b7bf-3f7966b4a9b5 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.799449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07fa7b-0aad-4182-b7bf-3f7966b4a9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.808317 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511158b-adf8-42ba-8269-55b6061d6406 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:37.810041 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511158b-adf8-42ba-8269-55b6061d6406 service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:37.823602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1358/2760] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:37.830338 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-2c07fa7b-0aad-4182-b7bf-3f7966b4a9b5 None None] MFA Rules not processed for user `d1b01c21c38949938141eefdc4939385`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:37.831574 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2c07fa7b-0aad-4182-b7bf-3f7966b4a9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:37.842374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1403/2761] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Aug 26 16:00:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:00:37.857187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511158b-adf8-42ba-8269-55b6061d6406 service nova] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:37.880263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511158b-adf8-42ba-8269-55b6061d6406 service nova] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:37.887278 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179139cf-c387-4850-8efd-d32025a27317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:37.888040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179139cf-c387-4850-8efd-d32025a27317 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.888422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179139cf-c387-4850-8efd-d32025a27317 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.888705 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179139cf-c387-4850-8efd-d32025a27317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.902493 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179139cf-c387-4850-8efd-d32025a27317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:37.904896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179139cf-c387-4850-8efd-d32025a27317 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:37.923192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1359/2762] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:38.003096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07fa7b-0aad-4182-b7bf-3f7966b4a9b5 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:38.005691 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2c07fa7b-0aad-4182-b7bf-3f7966b4a9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:38.092735 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:38.100173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07fa7b-0aad-4182-b7bf-3f7966b4a9b5 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:38.111668 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da1775-fd2a-475f-b3cd-7704fb901ad2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:38.112489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da1775-fd2a-475f-b3cd-7704fb901ad2 service placement] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:38.112771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da1775-fd2a-475f-b3cd-7704fb901ad2 service placement] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:38.113111 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da1775-fd2a-475f-b3cd-7704fb901ad2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:38.115355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f7da1775-fd2a-475f-b3cd-7704fb901ad2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:38.131514 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da1775-fd2a-475f-b3cd-7704fb901ad2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:38.133184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da1775-fd2a-475f-b3cd-7704fb901ad2 service placement] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:38.146569 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1404/2763] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:40.740126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4c76-8cf9-46e7-a597-7421b64b225d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:40.740712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4c76-8cf9-46e7-a597-7421b64b225d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:40.740712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4c76-8cf9-46e7-a597-7421b64b225d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:40.804566 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-96ef4c76-8cf9-46e7-a597-7421b64b225d None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:40.806407 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-96ef4c76-8cf9-46e7-a597-7421b64b225d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:40.916027 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1360/2764] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 16:00:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:40.943008 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da1775-fd2a-475f-b3cd-7704fb901ad2 service placement] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:40.946901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f7da1775-fd2a-475f-b3cd-7704fb901ad2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:41.078012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da1775-fd2a-475f-b3cd-7704fb901ad2 service placement] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:41.083605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ce473-4cab-4e13-9e75-506d15fc29f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:41.084252 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ce473-4cab-4e13-9e75-506d15fc29f7 service swift] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:41.084539 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ce473-4cab-4e13-9e75-506d15fc29f7 service swift] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:41.084815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ce473-4cab-4e13-9e75-506d15fc29f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:41.100629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ce473-4cab-4e13-9e75-506d15fc29f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:41.104738 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ce473-4cab-4e13-9e75-506d15fc29f7 service swift] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:41.111986 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1405/2765] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 16:00:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:00:42.767928 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef4c76-8cf9-46e7-a597-7421b64b225d None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:42.769700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-96ef4c76-8cf9-46e7-a597-7421b64b225d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:42.785929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef4c76-8cf9-46e7-a597-7421b64b225d None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:42.790367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b86c50-9f05-4b4d-97c0-23ea896162a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:42.791127 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b86c50-9f05-4b4d-97c0-23ea896162a7 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:42.791474 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b86c50-9f05-4b4d-97c0-23ea896162a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:42.791753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b86c50-9f05-4b4d-97c0-23ea896162a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:42.793816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a3b86c50-9f05-4b4d-97c0-23ea896162a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:42.833312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=100 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:42.865985 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b86c50-9f05-4b4d-97c0-23ea896162a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:42.868934 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b86c50-9f05-4b4d-97c0-23ea896162a7 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:42.881090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1361/2766] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:51.107771 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=46 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:52.879674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=15 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:53.431854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ce473-4cab-4e13-9e75-506d15fc29f7 service swift] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:53.433555 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0c0ce473-4cab-4e13-9e75-506d15fc29f7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:53.448280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ce473-4cab-4e13-9e75-506d15fc29f7 service swift] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:53.454621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:53.455288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:53.455546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:53.456066 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] PATH_INFO: `/v3/users/436adf49f3e24e03971017ea6dd33ac8` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:53.457232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] RBAC: Authorizing `identity:delete_user(user_id=436adf49f3e24e03971017ea6dd33ac8)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:53.459164 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:53.637868 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436adf49f3e24e03971017ea6dd33ac8'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:53.770884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436adf49f3e24e03971017ea6dd33ac8'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:53.770884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436adf49f3e24e03971017ea6dd33ac8'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:53.770884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436adf49f3e24e03971017ea6dd33ac8'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:53.863108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436adf49f3e24e03971017ea6dd33ac8'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:53.876377 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1406/2767] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:00:53 2022] DELETE /identity/v3/users/436adf49f3e24e03971017ea6dd33ac8 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:00:54.145191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b86c50-9f05-4b4d-97c0-23ea896162a7 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:54.156125 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b86c50-9f05-4b4d-97c0-23ea896162a7 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:54.216001 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:54.216646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:54.216929 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:54.217208 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] PATH_INFO: `/v3/projects/d19b01b0af6c43feb357799d8ae948e1` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:54.218330 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d19b01b0af6c43feb357799d8ae948e1)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:54.220360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:54.322984 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19b01b0af6c43feb357799d8ae948e1', 'request_id': 'req-fa7cc966-1991-4516-909b-9b4f70b77247'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:54.328753 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19b01b0af6c43feb357799d8ae948e1', 'request_id': 'req-fa7cc966-1991-4516-909b-9b4f70b77247'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:54.335265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19b01b0af6c43feb357799d8ae948e1', 'request_id': 'req-fa7cc966-1991-4516-909b-9b4f70b77247'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:54.423976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] The token cache is being invalidate because project d19b01b0af6c43feb357799d8ae948e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:00:54.428562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:54.428562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:54.428562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1362/2768] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:00:54 2022] DELETE /identity/v3/projects/d19b01b0af6c43feb357799d8ae948e1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:00:54.433122 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:54.435415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:54.454231 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231706f-8e9b-41fc-9ffc-90bc39093def admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:54.557910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa4ca3-e66e-4e01-b59e-8504213f37a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:54.558879 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa4ca3-e66e-4e01-b59e-8504213f37a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:54.559343 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa4ca3-e66e-4e01-b59e-8504213f37a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:54.559798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa4ca3-e66e-4e01-b59e-8504213f37a5 admin admin] PATH_INFO: `/v3/users/99e270149d4a42849cd73dd1267f15c0` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:54.561680 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa4ca3-e66e-4e01-b59e-8504213f37a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e270149d4a42849cd73dd1267f15c0)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:54.567095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa4ca3-e66e-4e01-b59e-8504213f37a5 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:54.848113 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-60fa4ca3-e66e-4e01-b59e-8504213f37a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e270149d4a42849cd73dd1267f15c0'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:54.865213 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-60fa4ca3-e66e-4e01-b59e-8504213f37a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e270149d4a42849cd73dd1267f15c0'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:54.866139 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-60fa4ca3-e66e-4e01-b59e-8504213f37a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e270149d4a42849cd73dd1267f15c0'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:54.866758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-60fa4ca3-e66e-4e01-b59e-8504213f37a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e270149d4a42849cd73dd1267f15c0'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:54.959788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-60fa4ca3-e66e-4e01-b59e-8504213f37a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e270149d4a42849cd73dd1267f15c0'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:54.963058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:54.974380 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1407/2769] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:00:54 2022] DELETE /identity/v3/users/99e270149d4a42849cd73dd1267f15c0 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:00:55.253984 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:55.256750 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:55.272465 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7cc966-1991-4516-909b-9b4f70b77247 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:55.344941 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:55.345576 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:55.345863 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:55.346142 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] PATH_INFO: `/v3/projects/ef6bcceb1f214427bbd4a07c5aceb0ed` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:55.347350 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6bcceb1f214427bbd4a07c5aceb0ed)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:55.349307 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:55.463451 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:55.476037 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6bcceb1f214427bbd4a07c5aceb0ed', 'request_id': 'req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:55.482512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6bcceb1f214427bbd4a07c5aceb0ed', 'request_id': 'req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:55.488860 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6bcceb1f214427bbd4a07c5aceb0ed', 'request_id': 'req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:55.578095 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] The token cache is being invalidate because project ef6bcceb1f214427bbd4a07c5aceb0ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:00:55.578316 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:55.579021 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:55.580831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1363/2770] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:00:55 2022] DELETE /identity/v3/projects/ef6bcceb1f214427bbd4a07c5aceb0ed => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:01.222085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aff1ac-b665-4a32-978d-6e4e4df2a640 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:01.222572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aff1ac-b665-4a32-978d-6e4e4df2a640 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:01.222659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aff1ac-b665-4a32-978d-6e4e4df2a640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:01.245836 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-35aff1ac-b665-4a32-978d-6e4e4df2a640 None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:01.247056 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-35aff1ac-b665-4a32-978d-6e4e4df2a640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:01.307755 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1408/2771] 10.222.0.98 () {58 vars in 967 bytes} [Fri Aug 26 16:01:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:01.322325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:01.323789 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:01.406338 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657d63c-d157-4820-8a8c-1ecc7c7af5b1 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:01.411542 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2de173-d640-48d7-a63c-4f629a99542c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:01.412216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de173-d640-48d7-a63c-4f629a99542c service swift] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:01.412504 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de173-d640-48d7-a63c-4f629a99542c service swift] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:01.412786 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de173-d640-48d7-a63c-4f629a99542c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:01.423320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2de173-d640-48d7-a63c-4f629a99542c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:01.425029 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2de173-d640-48d7-a63c-4f629a99542c service swift] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:01.435567 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1364/2772] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 16:01:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:01:07.406537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aff1ac-b665-4a32-978d-6e4e4df2a640 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:07.408096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-35aff1ac-b665-4a32-978d-6e4e4df2a640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:07.475424 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aff1ac-b665-4a32-978d-6e4e4df2a640 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:07.481053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:07.481698 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:07.481980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:07.482257 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] PATH_INFO: `/v3/users/75f57356920a4d6ab3277bb0931349cc` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:07.483428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75f57356920a4d6ab3277bb0931349cc)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:07.485387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:07.659307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75f57356920a4d6ab3277bb0931349cc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:07.671268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75f57356920a4d6ab3277bb0931349cc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:07.672081 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75f57356920a4d6ab3277bb0931349cc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:07.672692 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75f57356920a4d6ab3277bb0931349cc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:07.766171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75f57356920a4d6ab3277bb0931349cc'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:07.781108 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1409/2773] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:01:07 2022] DELETE /identity/v3/users/75f57356920a4d6ab3277bb0931349cc => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:07.785755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94abb3f-8561-429f-b657-adc13739e4f6 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:07.786426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94abb3f-8561-429f-b657-adc13739e4f6 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:07.786426 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94abb3f-8561-429f-b657-adc13739e4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:07.807853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-d94abb3f-8561-429f-b657-adc13739e4f6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:07.809163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d94abb3f-8561-429f-b657-adc13739e4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:07.873163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1365/2774] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:07.883248 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:07.884756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:07.904545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:07.945538 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10be47-886c-482b-8e2b-a9aa41267a40 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:07.950731 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604937d-f8b9-464c-af54-14b82e8398af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:07.951486 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604937d-f8b9-464c-af54-14b82e8398af service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:07.951987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604937d-f8b9-464c-af54-14b82e8398af service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:07.952490 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604937d-f8b9-464c-af54-14b82e8398af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:07.964308 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604937d-f8b9-464c-af54-14b82e8398af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:07.965756 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604937d-f8b9-464c-af54-14b82e8398af service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:07.975428 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1410/2775] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:08.453220 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94abb3f-8561-429f-b657-adc13739e4f6 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:08.454674 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d94abb3f-8561-429f-b657-adc13739e4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:08.466793 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94abb3f-8561-429f-b657-adc13739e4f6 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:08.471425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:08.472100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:08.472381 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:08.472708 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] PATH_INFO: `/v3/projects/4d19ae3de8c54cbaa0317ea4950f0450` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:08.473862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d19ae3de8c54cbaa0317ea4950f0450)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:08.476292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:08.587541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:08.590402 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d19ae3de8c54cbaa0317ea4950f0450', 'request_id': 'req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:08.596000 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d19ae3de8c54cbaa0317ea4950f0450', 'request_id': 'req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:08.602946 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d19ae3de8c54cbaa0317ea4950f0450', 'request_id': 'req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:08.684935 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] The token cache is being invalidate because project 4d19ae3de8c54cbaa0317ea4950f0450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:01:08.685205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:08.685847 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:08.687312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1366/2776] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:01:08 2022] DELETE /identity/v3/projects/4d19ae3de8c54cbaa0317ea4950f0450 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:18.005058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=19 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:18.737265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:32.746596 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604937d-f8b9-464c-af54-14b82e8398af service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:32.748102 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-4604937d-f8b9-464c-af54-14b82e8398af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:32.806484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604937d-f8b9-464c-af54-14b82e8398af service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:32.859713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e6913-da20-473a-8cfe-7164117dedf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:32.859713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e6913-da20-473a-8cfe-7164117dedf6 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:32.859713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e6913-da20-473a-8cfe-7164117dedf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:32.859713 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e6913-da20-473a-8cfe-7164117dedf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:32.860259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c38e6913-da20-473a-8cfe-7164117dedf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:32.919787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e6913-da20-473a-8cfe-7164117dedf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:32.921263 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e6913-da20-473a-8cfe-7164117dedf6 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:32.930437 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1411/2777] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:41.840367 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:41.841907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:41.901647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f901e-c4bc-4ca9-8ffc-4f3f842626b4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:41.905343 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb3ac1-d426-4625-99d5-910d047519ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:41.906005 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb3ac1-d426-4625-99d5-910d047519ae service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:41.906287 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb3ac1-d426-4625-99d5-910d047519ae service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:41.906560 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb3ac1-d426-4625-99d5-910d047519ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:41.908343 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-78cb3ac1-d426-4625-99d5-910d047519ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:41.972156 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb3ac1-d426-4625-99d5-910d047519ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:41.973610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb3ac1-d426-4625-99d5-910d047519ae service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:41.991217 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1367/2778] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:42.926320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=68 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:45.894444 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e6913-da20-473a-8cfe-7164117dedf6 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:45.898211 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c38e6913-da20-473a-8cfe-7164117dedf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:45.917910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e6913-da20-473a-8cfe-7164117dedf6 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:45.922588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef53bc5-108a-4921-a1fe-9e28f2282d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:45.923228 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef53bc5-108a-4921-a1fe-9e28f2282d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:45.923501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef53bc5-108a-4921-a1fe-9e28f2282d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:45.923776 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef53bc5-108a-4921-a1fe-9e28f2282d67 admin admin] PATH_INFO: `/v3/users/91966686fa7f4c5bb326cdb1e2abc1d9` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:45.924930 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef53bc5-108a-4921-a1fe-9e28f2282d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91966686fa7f4c5bb326cdb1e2abc1d9)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:45.926921 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef53bc5-108a-4921-a1fe-9e28f2282d67 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:46.117753 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-fef53bc5-108a-4921-a1fe-9e28f2282d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91966686fa7f4c5bb326cdb1e2abc1d9'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:46.131074 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-fef53bc5-108a-4921-a1fe-9e28f2282d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91966686fa7f4c5bb326cdb1e2abc1d9'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:46.131829 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-fef53bc5-108a-4921-a1fe-9e28f2282d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91966686fa7f4c5bb326cdb1e2abc1d9'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:46.132482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-fef53bc5-108a-4921-a1fe-9e28f2282d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91966686fa7f4c5bb326cdb1e2abc1d9'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:46.210674 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-fef53bc5-108a-4921-a1fe-9e28f2282d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91966686fa7f4c5bb326cdb1e2abc1d9'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:46.225290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1412/2779] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:01:45 2022] DELETE /identity/v3/users/91966686fa7f4c5bb326cdb1e2abc1d9 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:46.514936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb3ac1-d426-4625-99d5-910d047519ae service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:46.529089 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb3ac1-d426-4625-99d5-910d047519ae service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:46.590193 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:46.590833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:46.591118 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:46.591395 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] PATH_INFO: `/v3/projects/10b0a72955244448954be539f99435e9` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:46.592546 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b0a72955244448954be539f99435e9)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:46.594467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:46.714827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b0a72955244448954be539f99435e9', 'request_id': 'req-aca21d04-c961-4088-9cf7-495bf786f072'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:46.716800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:46.720796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b0a72955244448954be539f99435e9', 'request_id': 'req-aca21d04-c961-4088-9cf7-495bf786f072'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:46.727044 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b0a72955244448954be539f99435e9', 'request_id': 'req-aca21d04-c961-4088-9cf7-495bf786f072'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:46.838647 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] The token cache is being invalidate because project 10b0a72955244448954be539f99435e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:01:46.838902 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:46.839470 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-aca21d04-c961-4088-9cf7-495bf786f072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:46.841040 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1368/2780] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:01:46 2022] DELETE /identity/v3/projects/10b0a72955244448954be539f99435e9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:46.846001 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c89684-731d-4693-adb4-2ce2f942f3b2 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:46.846295 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c89684-731d-4693-adb4-2ce2f942f3b2 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:46.846577 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c89684-731d-4693-adb4-2ce2f942f3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:46.868457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-b1c89684-731d-4693-adb4-2ce2f942f3b2 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:46.869674 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-b1c89684-731d-4693-adb4-2ce2f942f3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:46.930808 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1413/2781] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:46.938842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4550c-aa2d-4d01-b8c2-6685022ec33a None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:46.939130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4550c-aa2d-4d01-b8c2-6685022ec33a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:46.939467 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4550c-aa2d-4d01-b8c2-6685022ec33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:46.960787 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-5cf4550c-aa2d-4d01-b8c2-6685022ec33a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:46.962026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5cf4550c-aa2d-4d01-b8c2-6685022ec33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:46.970069 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1369/2782] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:46.975142 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c89684-731d-4693-adb4-2ce2f942f3b2 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:46.985173 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c89684-731d-4693-adb4-2ce2f942f3b2 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:46.990012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7b7bd-15f5-4ad0-b900-b6a83fa70938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:46.990614 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7b7bd-15f5-4ad0-b900-b6a83fa70938 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:46.990898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7b7bd-15f5-4ad0-b900-b6a83fa70938 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:46.991177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7b7bd-15f5-4ad0-b900-b6a83fa70938 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:46.991862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7b7bd-15f5-4ad0-b900-b6a83fa70938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:01:46.992184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7b7bd-15f5-4ad0-b900-b6a83fa70938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:46.993787 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7b7bd-15f5-4ad0-b900-b6a83fa70938 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:47.005708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1414/2783] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 16:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:01:47.010229 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4550c-aa2d-4d01-b8c2-6685022ec33a None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:47.019718 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4550c-aa2d-4d01-b8c2-6685022ec33a None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:47.025256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2f04f-0f85-4e25-b66f-247e27650c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:47.025898 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2f04f-0f85-4e25-b66f-247e27650c1d admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:47.026187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2f04f-0f85-4e25-b66f-247e27650c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:47.026468 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2f04f-0f85-4e25-b66f-247e27650c1d admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:47.027114 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2f04f-0f85-4e25-b66f-247e27650c1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:47.029130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2f04f-0f85-4e25-b66f-247e27650c1d admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:47.166560 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1370/2784] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 16:01:47 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:47.171645 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7b7bd-15f5-4ad0-b900-b6a83fa70938 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:47.187117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7b7bd-15f5-4ad0-b900-b6a83fa70938 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:47.246670 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab91d1c-ca0f-4914-b3c2-cc8861f06694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:47.247297 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab91d1c-ca0f-4914-b3c2-cc8861f06694 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:47.247567 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab91d1c-ca0f-4914-b3c2-cc8861f06694 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:47.247862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab91d1c-ca0f-4914-b3c2-cc8861f06694 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:47.248575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab91d1c-ca0f-4914-b3c2-cc8861f06694 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:47.250536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab91d1c-ca0f-4914-b3c2-cc8861f06694 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:47.258603 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:47.371049 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1415/2785] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 16:01:47 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:47.375428 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2f04f-0f85-4e25-b66f-247e27650c1d admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:47.386437 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2f04f-0f85-4e25-b66f-247e27650c1d admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:47.391120 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c74abb-1397-4174-924e-e14c82236299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:47.391722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c74abb-1397-4174-924e-e14c82236299 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:47.392020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c74abb-1397-4174-924e-e14c82236299 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:47.392300 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c74abb-1397-4174-924e-e14c82236299 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:47.392945 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c74abb-1397-4174-924e-e14c82236299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:47.394663 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c74abb-1397-4174-924e-e14c82236299 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:47.404802 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1371/2786] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:01:47.413886 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab91d1c-ca0f-4914-b3c2-cc8861f06694 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:47.424087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab91d1c-ca0f-4914-b3c2-cc8861f06694 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:47.429370 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1bd71-9a00-4443-8bf1-de31729b1507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:47.429959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1bd71-9a00-4443-8bf1-de31729b1507 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:47.430243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1bd71-9a00-4443-8bf1-de31729b1507 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:47.430519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1bd71-9a00-4443-8bf1-de31729b1507 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:47.431191 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1bd71-9a00-4443-8bf1-de31729b1507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:47.432814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1bd71-9a00-4443-8bf1-de31729b1507 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:47.444295 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1416/2787] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:01:47.448794 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c74abb-1397-4174-924e-e14c82236299 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:47.461222 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c74abb-1397-4174-924e-e14c82236299 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:47.465865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3123600-8e24-4ed2-8cb9-e67016a81a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:47.466521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3123600-8e24-4ed2-8cb9-e67016a81a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:47.466838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3123600-8e24-4ed2-8cb9-e67016a81a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:47.467113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3123600-8e24-4ed2-8cb9-e67016a81a88 admin admin] PATH_INFO: `/v3/projects/02c5f910161c420e8d2c3efc8d633dde/users/f0889f8e44154cbe913c7b18f09d8488/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:47.482646 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3123600-8e24-4ed2-8cb9-e67016a81a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c5f910161c420e8d2c3efc8d633dde, user_id=f0889f8e44154cbe913c7b18f09d8488, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:47.485355 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3123600-8e24-4ed2-8cb9-e67016a81a88 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:47.622692 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1372/2788] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:01:47 2022] PUT /identity/v3/projects/02c5f910161c420e8d2c3efc8d633dde/users/f0889f8e44154cbe913c7b18f09d8488/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:01:47.634784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8efd9-36ae-41e3-8900-882f7e3150d9 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:47.635075 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8efd9-36ae-41e3-8900-882f7e3150d9 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:47.635355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8efd9-36ae-41e3-8900-882f7e3150d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:47.663425 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-c9c8efd9-36ae-41e3-8900-882f7e3150d9 None None] MFA Rules not processed for user `f0889f8e44154cbe913c7b18f09d8488`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:47.664726 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c9c8efd9-36ae-41e3-8900-882f7e3150d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:47.736942 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1417/2789] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:47.741954 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3123600-8e24-4ed2-8cb9-e67016a81a88 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:47.806143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3123600-8e24-4ed2-8cb9-e67016a81a88 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:47.810905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364784cb-4d15-4566-9217-ec4a27250bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:47.811516 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364784cb-4d15-4566-9217-ec4a27250bcd admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:47.811797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364784cb-4d15-4566-9217-ec4a27250bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:47.812102 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364784cb-4d15-4566-9217-ec4a27250bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:47.812739 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364784cb-4d15-4566-9217-ec4a27250bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:47.814690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364784cb-4d15-4566-9217-ec4a27250bcd admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:47.904051 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1373/2790] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 16:01:47 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:47.908184 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8efd9-36ae-41e3-8900-882f7e3150d9 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:47.919642 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8efd9-36ae-41e3-8900-882f7e3150d9 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:47.977586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550dd76-9d66-4be9-94f4-b5e21a46f2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:47.978264 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550dd76-9d66-4be9-94f4-b5e21a46f2fe admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:47.978465 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550dd76-9d66-4be9-94f4-b5e21a46f2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:47.978739 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550dd76-9d66-4be9-94f4-b5e21a46f2fe admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:47.979371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550dd76-9d66-4be9-94f4-b5e21a46f2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:47.981489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550dd76-9d66-4be9-94f4-b5e21a46f2fe admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:48.074509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1418/2791] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 16:01:47 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:48.078642 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364784cb-4d15-4566-9217-ec4a27250bcd admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:48.089863 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364784cb-4d15-4566-9217-ec4a27250bcd admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:48.094593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74863754-2fde-4533-b046-0747121d9205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:48.095205 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74863754-2fde-4533-b046-0747121d9205 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:48.095490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74863754-2fde-4533-b046-0747121d9205 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:48.095767 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74863754-2fde-4533-b046-0747121d9205 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:48.096439 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74863754-2fde-4533-b046-0747121d9205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:48.098033 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74863754-2fde-4533-b046-0747121d9205 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:48.109490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1374/2792] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:01:48.114074 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550dd76-9d66-4be9-94f4-b5e21a46f2fe admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:48.125175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550dd76-9d66-4be9-94f4-b5e21a46f2fe admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:48.130209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aafe52-7cbb-40f9-a339-b14e6031c483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:48.130872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aafe52-7cbb-40f9-a339-b14e6031c483 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:48.131185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aafe52-7cbb-40f9-a339-b14e6031c483 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:48.131472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aafe52-7cbb-40f9-a339-b14e6031c483 admin admin] PATH_INFO: `/v3/projects/985bae11d2af4042942e15b799223181/users/2ffd31850d0b4f9cb5af463ea5e0c348/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:48.146989 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aafe52-7cbb-40f9-a339-b14e6031c483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985bae11d2af4042942e15b799223181, user_id=2ffd31850d0b4f9cb5af463ea5e0c348, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:48.149666 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aafe52-7cbb-40f9-a339-b14e6031c483 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:48.222625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1419/2793] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:01:48 2022] PUT /identity/v3/projects/985bae11d2af4042942e15b799223181/users/2ffd31850d0b4f9cb5af463ea5e0c348/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:48.227304 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74863754-2fde-4533-b046-0747121d9205 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:48.237457 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74863754-2fde-4533-b046-0747121d9205 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:48.284899 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:48.295580 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9f1ad-e150-44f9-9b00-6979c7bfa44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:48.296219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9f1ad-e150-44f9-9b00-6979c7bfa44f admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:48.296505 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9f1ad-e150-44f9-9b00-6979c7bfa44f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:48.296783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9f1ad-e150-44f9-9b00-6979c7bfa44f admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:48.297447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9f1ad-e150-44f9-9b00-6979c7bfa44f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:48.299164 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9f1ad-e150-44f9-9b00-6979c7bfa44f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:48.309521 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1375/2794] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:01:48.313864 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aafe52-7cbb-40f9-a339-b14e6031c483 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:48.332711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aafe52-7cbb-40f9-a339-b14e6031c483 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:48.332711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cc259-c21a-405f-a530-31c245a3f8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:48.332711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cc259-c21a-405f-a530-31c245a3f8cb admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:48.332711 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cc259-c21a-405f-a530-31c245a3f8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:48.333745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cc259-c21a-405f-a530-31c245a3f8cb admin admin] PATH_INFO: `/v3/projects/985bae11d2af4042942e15b799223181/users/2ffd31850d0b4f9cb5af463ea5e0c348/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:48.333745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cc259-c21a-405f-a530-31c245a3f8cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=985bae11d2af4042942e15b799223181, user_id=2ffd31850d0b4f9cb5af463ea5e0c348, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:48.336115 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cc259-c21a-405f-a530-31c245a3f8cb admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:48.346482 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1420/2795] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:01:48 2022] PUT /identity/v3/projects/985bae11d2af4042942e15b799223181/users/2ffd31850d0b4f9cb5af463ea5e0c348/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:48.356574 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1312595-0e68-4ab7-864c-e85db94b31cc None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:48.356862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1312595-0e68-4ab7-864c-e85db94b31cc None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:48.357170 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1312595-0e68-4ab7-864c-e85db94b31cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:48.385825 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-a1312595-0e68-4ab7-864c-e85db94b31cc None None] MFA Rules not processed for user `2ffd31850d0b4f9cb5af463ea5e0c348`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:48.387099 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-a1312595-0e68-4ab7-864c-e85db94b31cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:48.457827 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1376/2796] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:01:48.468748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cc259-c21a-405f-a530-31c245a3f8cb admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:48.470505 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6e5cc259-c21a-405f-a530-31c245a3f8cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:48.493834 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:48.530919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cc259-c21a-405f-a530-31c245a3f8cb admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:48.535129 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1945e9c-40b6-483a-a633-1cdeffaab6f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:48.535745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1945e9c-40b6-483a-a633-1cdeffaab6f2 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:48.539515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1945e9c-40b6-483a-a633-1cdeffaab6f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:48.539515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1945e9c-40b6-483a-a633-1cdeffaab6f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:48.545623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1945e9c-40b6-483a-a633-1cdeffaab6f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:48.547078 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1945e9c-40b6-483a-a633-1cdeffaab6f2 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:48.557177 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1421/2797] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:51.798755 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1312595-0e68-4ab7-864c-e85db94b31cc None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:51.862841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1312595-0e68-4ab7-864c-e85db94b31cc None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:51.867839 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:51.868541 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:51.868828 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:51.869112 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] PATH_INFO: `/v3/users/f0889f8e44154cbe913c7b18f09d8488` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:51.870332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0889f8e44154cbe913c7b18f09d8488)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:51.872354 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:52.076049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0889f8e44154cbe913c7b18f09d8488'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:52.076714 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0889f8e44154cbe913c7b18f09d8488'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:52.089310 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0889f8e44154cbe913c7b18f09d8488'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:52.176549 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0889f8e44154cbe913c7b18f09d8488'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:52.181425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:52.187260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0889f8e44154cbe913c7b18f09d8488'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:52.194157 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1377/2798] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:01:51 2022] DELETE /identity/v3/users/f0889f8e44154cbe913c7b18f09d8488 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:52.198699 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7ea4d-1649-48fc-9ee0-2e5500e64fe8 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:52.199185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7ea4d-1649-48fc-9ee0-2e5500e64fe8 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:52.199296 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7ea4d-1649-48fc-9ee0-2e5500e64fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:52.218057 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-13f7ea4d-1649-48fc-9ee0-2e5500e64fe8 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:52.219279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-13f7ea4d-1649-48fc-9ee0-2e5500e64fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:52.279109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1422/2799] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:52.289679 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:52.291155 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:52.350388 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29e84f-57d3-421b-93fd-25cd78a259c6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:52.354121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba74d20-b295-40c6-85fa-044e33c5e2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:52.354771 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74d20-b295-40c6-85fa-044e33c5e2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:52.355133 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74d20-b295-40c6-85fa-044e33c5e2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:52.355409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74d20-b295-40c6-85fa-044e33c5e2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:52.364907 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba74d20-b295-40c6-85fa-044e33c5e2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:52.366436 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba74d20-b295-40c6-85fa-044e33c5e2c2 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:52.375999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1378/2800] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:52.846384 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7ea4d-1649-48fc-9ee0-2e5500e64fe8 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:52.847963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-13f7ea4d-1649-48fc-9ee0-2e5500e64fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:52.867004 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7ea4d-1649-48fc-9ee0-2e5500e64fe8 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:52.872751 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:52.873421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:52.873714 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:52.874007 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] PATH_INFO: `/v3/projects/02c5f910161c420e8d2c3efc8d633dde` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:52.875312 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c5f910161c420e8d2c3efc8d633dde)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:52.877546 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:52.993876 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c5f910161c420e8d2c3efc8d633dde', 'request_id': 'req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:53.067529 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c5f910161c420e8d2c3efc8d633dde', 'request_id': 'req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:53.074768 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c5f910161c420e8d2c3efc8d633dde', 'request_id': 'req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:53.086266 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] The token cache is being invalidate because project 02c5f910161c420e8d2c3efc8d633dde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:01:53.086849 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:53.087385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:53.089036 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1423/2801] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:01:52 2022] DELETE /identity/v3/projects/02c5f910161c420e8d2c3efc8d633dde => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:53.093018 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba74d20-b295-40c6-85fa-044e33c5e2c2 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:53.103896 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba74d20-b295-40c6-85fa-044e33c5e2c2 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:53.162219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92b7df-c9af-408b-bfa7-8be46d3e7ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:53.162893 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b7df-c9af-408b-bfa7-8be46d3e7ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:53.163203 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b7df-c9af-408b-bfa7-8be46d3e7ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:53.163573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b7df-c9af-408b-bfa7-8be46d3e7ff0 admin admin] PATH_INFO: `/v3/users/2ffd31850d0b4f9cb5af463ea5e0c348` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:53.166194 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92b7df-c9af-408b-bfa7-8be46d3e7ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffd31850d0b4f9cb5af463ea5e0c348)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:53.168154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92b7df-c9af-408b-bfa7-8be46d3e7ff0 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:53.349903 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fd92b7df-c9af-408b-bfa7-8be46d3e7ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffd31850d0b4f9cb5af463ea5e0c348'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:53.350529 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fd92b7df-c9af-408b-bfa7-8be46d3e7ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffd31850d0b4f9cb5af463ea5e0c348'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:53.362603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fd92b7df-c9af-408b-bfa7-8be46d3e7ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffd31850d0b4f9cb5af463ea5e0c348'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:53.437511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fd92b7df-c9af-408b-bfa7-8be46d3e7ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffd31850d0b4f9cb5af463ea5e0c348'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:53.445589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-fd92b7df-c9af-408b-bfa7-8be46d3e7ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffd31850d0b4f9cb5af463ea5e0c348'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:53.451833 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1379/2802] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:01:53 2022] DELETE /identity/v3/users/2ffd31850d0b4f9cb5af463ea5e0c348 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:53.914562 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:53.916188 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:53.934401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6575ca8-9fba-4b6a-a5d2-dc24459521e8 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:53.982557 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:53.995671 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:53.996348 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:53.996635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:53.996916 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] PATH_INFO: `/v3/projects/985bae11d2af4042942e15b799223181` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:53.998070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=985bae11d2af4042942e15b799223181)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:54.000001 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:54.120874 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985bae11d2af4042942e15b799223181', 'request_id': 'req-1abe7013-66b3-45fe-b85a-4d6325ea59fa'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:54.226257 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985bae11d2af4042942e15b799223181', 'request_id': 'req-1abe7013-66b3-45fe-b85a-4d6325ea59fa'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:54.233451 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985bae11d2af4042942e15b799223181', 'request_id': 'req-1abe7013-66b3-45fe-b85a-4d6325ea59fa'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:54.244095 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] The token cache is being invalidate because project 985bae11d2af4042942e15b799223181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:01:54.244387 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:54.245070 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-1abe7013-66b3-45fe-b85a-4d6325ea59fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:54.246551 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1424/2803] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:01:53 2022] DELETE /identity/v3/projects/985bae11d2af4042942e15b799223181 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:54.251511 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f605d-d908-4682-9eb4-5e232000224b None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:54.252049 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f605d-d908-4682-9eb4-5e232000224b None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:54.252161 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f605d-d908-4682-9eb4-5e232000224b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:54.273736 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-929f605d-d908-4682-9eb4-5e232000224b None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:54.274956 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-929f605d-d908-4682-9eb4-5e232000224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:54.285132 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:54.337291 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1380/2804] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:54.345221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73eb317-65cc-4bd6-8e1c-91ce324bf3d6 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:54.345540 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73eb317-65cc-4bd6-8e1c-91ce324bf3d6 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:54.345824 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73eb317-65cc-4bd6-8e1c-91ce324bf3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:54.366677 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e73eb317-65cc-4bd6-8e1c-91ce324bf3d6 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:54.367914 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e73eb317-65cc-4bd6-8e1c-91ce324bf3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:54.375999 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1425/2805] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:54.380585 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f605d-d908-4682-9eb4-5e232000224b None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:54.390633 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f605d-d908-4682-9eb4-5e232000224b None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:54.395862 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08a3bc-067c-432c-9d16-db55d1ea41a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:54.396530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08a3bc-067c-432c-9d16-db55d1ea41a6 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:54.396816 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08a3bc-067c-432c-9d16-db55d1ea41a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:54.397091 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08a3bc-067c-432c-9d16-db55d1ea41a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:54.397748 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08a3bc-067c-432c-9d16-db55d1ea41a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:01:54.398072 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08a3bc-067c-432c-9d16-db55d1ea41a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:54.399742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08a3bc-067c-432c-9d16-db55d1ea41a6 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:54.413145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1381/2806] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 16:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 16:01:54.417605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73eb317-65cc-4bd6-8e1c-91ce324bf3d6 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:54.427515 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73eb317-65cc-4bd6-8e1c-91ce324bf3d6 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:54.432323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d647f0-8965-4beb-95df-9cd7a05003d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:54.432927 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d647f0-8965-4beb-95df-9cd7a05003d3 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:54.433214 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d647f0-8965-4beb-95df-9cd7a05003d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:54.433496 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d647f0-8965-4beb-95df-9cd7a05003d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:54.434157 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d647f0-8965-4beb-95df-9cd7a05003d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:54.436096 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d647f0-8965-4beb-95df-9cd7a05003d3 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:54.520712 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1426/2807] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 16:01:54 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:54.524841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08a3bc-067c-432c-9d16-db55d1ea41a6 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:54.535178 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08a3bc-067c-432c-9d16-db55d1ea41a6 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:54.593996 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb9b10-c503-4c66-bd4f-3842c6be6c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:54.594619 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb9b10-c503-4c66-bd4f-3842c6be6c5a admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:54.595101 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb9b10-c503-4c66-bd4f-3842c6be6c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:54.595384 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb9b10-c503-4c66-bd4f-3842c6be6c5a admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:54.596085 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb9b10-c503-4c66-bd4f-3842c6be6c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:54.598246 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb9b10-c503-4c66-bd4f-3842c6be6c5a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:54.777003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1382/2808] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 16:01:54 2022] POST /identity/v3/users => generated 291 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:01:54.783588 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d647f0-8965-4beb-95df-9cd7a05003d3 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:54.792644 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d647f0-8965-4beb-95df-9cd7a05003d3 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:54.797275 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00291ae-bc34-491e-8d02-6db8cf4f670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:54.797873 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00291ae-bc34-491e-8d02-6db8cf4f670d admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:54.798159 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00291ae-bc34-491e-8d02-6db8cf4f670d admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:54.798499 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00291ae-bc34-491e-8d02-6db8cf4f670d admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:54.799150 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00291ae-bc34-491e-8d02-6db8cf4f670d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:54.800792 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00291ae-bc34-491e-8d02-6db8cf4f670d admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:54.811568 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1427/2809] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:01:54.815582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb9b10-c503-4c66-bd4f-3842c6be6c5a admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:54.826482 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb9b10-c503-4c66-bd4f-3842c6be6c5a admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:54.831455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c1f6f-4ed3-4e2b-968e-a03a3a724fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:54.832045 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c1f6f-4ed3-4e2b-968e-a03a3a724fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:54.832332 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c1f6f-4ed3-4e2b-968e-a03a3a724fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:54.832610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c1f6f-4ed3-4e2b-968e-a03a3a724fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:54.833263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c1f6f-4ed3-4e2b-968e-a03a3a724fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:54.834906 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c1f6f-4ed3-4e2b-968e-a03a3a724fd3 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:54.845906 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1383/2810] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:01:54.850799 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00291ae-bc34-491e-8d02-6db8cf4f670d admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:54.861175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00291ae-bc34-491e-8d02-6db8cf4f670d admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:54.866535 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1348a2a-492d-47a0-a89d-df93afe2b46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:54.867189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1348a2a-492d-47a0-a89d-df93afe2b46f admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:54.867473 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1348a2a-492d-47a0-a89d-df93afe2b46f admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:54.867753 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1348a2a-492d-47a0-a89d-df93afe2b46f admin admin] PATH_INFO: `/v3/projects/3f7fa945bb1d4fa9aa2038cb14f89936/users/3a21475ebb1d45239f449ad643e772da/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:54.883017 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1348a2a-492d-47a0-a89d-df93afe2b46f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f7fa945bb1d4fa9aa2038cb14f89936, user_id=3a21475ebb1d45239f449ad643e772da, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:54.885741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1348a2a-492d-47a0-a89d-df93afe2b46f admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:55.005163 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1428/2811] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:01:54 2022] PUT /identity/v3/projects/3f7fa945bb1d4fa9aa2038cb14f89936/users/3a21475ebb1d45239f449ad643e772da/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:55.016860 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175de58b-2fd0-43c2-9e54-70b47ebb90ff None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:55.017148 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175de58b-2fd0-43c2-9e54-70b47ebb90ff None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:55.017480 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175de58b-2fd0-43c2-9e54-70b47ebb90ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:55.048245 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-175de58b-2fd0-43c2-9e54-70b47ebb90ff None None] MFA Rules not processed for user `3a21475ebb1d45239f449ad643e772da`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:55.049408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-175de58b-2fd0-43c2-9e54-70b47ebb90ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:55.120351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1384/2812] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:55.125040 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1348a2a-492d-47a0-a89d-df93afe2b46f admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:55.190059 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1348a2a-492d-47a0-a89d-df93afe2b46f admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:55.194832 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b941a0-39b5-4fa4-ab7d-552ef1ef069c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:55.195438 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b941a0-39b5-4fa4-ab7d-552ef1ef069c admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:55.196024 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b941a0-39b5-4fa4-ab7d-552ef1ef069c admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:55.196287 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b941a0-39b5-4fa4-ab7d-552ef1ef069c admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:55.196935 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b941a0-39b5-4fa4-ab7d-552ef1ef069c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:55.198918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b941a0-39b5-4fa4-ab7d-552ef1ef069c admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:55.330563 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:55.338418 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1429/2813] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 16:01:55 2022] POST /identity/v3/projects => generated 379 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:55.342639 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175de58b-2fd0-43c2-9e54-70b47ebb90ff None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:55.354445 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175de58b-2fd0-43c2-9e54-70b47ebb90ff None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:55.359168 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:55.410262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63099953-a275-408a-9156-2cf41c73642e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:55.410938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63099953-a275-408a-9156-2cf41c73642e admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:55.411716 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63099953-a275-408a-9156-2cf41c73642e admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:55.412031 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63099953-a275-408a-9156-2cf41c73642e admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:55.412675 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63099953-a275-408a-9156-2cf41c73642e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:55.414591 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63099953-a275-408a-9156-2cf41c73642e admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:55.513043 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1385/2814] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 16:01:55 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:55.516947 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b941a0-39b5-4fa4-ab7d-552ef1ef069c admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:55.528342 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b941a0-39b5-4fa4-ab7d-552ef1ef069c admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:55.533134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fafadb-3481-4336-b954-aab0e12f15da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:55.533759 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fafadb-3481-4336-b954-aab0e12f15da admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:55.534051 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fafadb-3481-4336-b954-aab0e12f15da admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:55.534332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fafadb-3481-4336-b954-aab0e12f15da admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:55.534984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fafadb-3481-4336-b954-aab0e12f15da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:55.536635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fafadb-3481-4336-b954-aab0e12f15da admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:55.547347 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1430/2815] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:01:55.551536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63099953-a275-408a-9156-2cf41c73642e admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:55.561880 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63099953-a275-408a-9156-2cf41c73642e admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:55.566687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef3380-e29b-405c-9604-944ad8a6004a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:55.568055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3380-e29b-405c-9604-944ad8a6004a admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:55.568447 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3380-e29b-405c-9604-944ad8a6004a admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:55.568725 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3380-e29b-405c-9604-944ad8a6004a admin admin] PATH_INFO: `/v3/projects/cda9e45c6bd749b680bbba30a9ef6b7f/users/52554d78e12843bca66d2dd10e0fa41e/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:55.586282 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef3380-e29b-405c-9604-944ad8a6004a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda9e45c6bd749b680bbba30a9ef6b7f, user_id=52554d78e12843bca66d2dd10e0fa41e, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:55.589024 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef3380-e29b-405c-9604-944ad8a6004a admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:55.719145 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1386/2816] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:01:55 2022] PUT /identity/v3/projects/cda9e45c6bd749b680bbba30a9ef6b7f/users/52554d78e12843bca66d2dd10e0fa41e/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:55.723221 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fafadb-3481-4336-b954-aab0e12f15da admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:55.734395 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fafadb-3481-4336-b954-aab0e12f15da admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:55.795185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12189f-a9ac-47d7-946c-f0dda2388300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:55.795821 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12189f-a9ac-47d7-946c-f0dda2388300 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:55.796147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12189f-a9ac-47d7-946c-f0dda2388300 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:55.796431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12189f-a9ac-47d7-946c-f0dda2388300 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:55.797107 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12189f-a9ac-47d7-946c-f0dda2388300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:55.798775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12189f-a9ac-47d7-946c-f0dda2388300 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:55.810543 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1431/2817] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:01:55.814916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef3380-e29b-405c-9604-944ad8a6004a admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:55.826181 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef3380-e29b-405c-9604-944ad8a6004a admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:55.831298 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ab893-89e2-4759-8491-33444d7fc0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:55.834260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ab893-89e2-4759-8491-33444d7fc0e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:55.834260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ab893-89e2-4759-8491-33444d7fc0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:55.834260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ab893-89e2-4759-8491-33444d7fc0e4 admin admin] PATH_INFO: `/v3/projects/cda9e45c6bd749b680bbba30a9ef6b7f/users/52554d78e12843bca66d2dd10e0fa41e/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:55.834608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ab893-89e2-4759-8491-33444d7fc0e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda9e45c6bd749b680bbba30a9ef6b7f, user_id=52554d78e12843bca66d2dd10e0fa41e, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:55.837404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ab893-89e2-4759-8491-33444d7fc0e4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:55.849247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1387/2818] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:01:55 2022] PUT /identity/v3/projects/cda9e45c6bd749b680bbba30a9ef6b7f/users/52554d78e12843bca66d2dd10e0fa41e/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:55.860011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e51f70-865a-4464-a586-bd8a0f2c90c2 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:55.860321 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e51f70-865a-4464-a586-bd8a0f2c90c2 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:55.860599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e51f70-865a-4464-a586-bd8a0f2c90c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:55.888883 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-62e51f70-865a-4464-a586-bd8a0f2c90c2 None None] MFA Rules not processed for user `52554d78e12843bca66d2dd10e0fa41e`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:55.890121 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-62e51f70-865a-4464-a586-bd8a0f2c90c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:55.958162 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1432/2819] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:55.969712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ab893-89e2-4759-8491-33444d7fc0e4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:55.971446 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5d1ab893-89e2-4759-8491-33444d7fc0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:56.037612 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ab893-89e2-4759-8491-33444d7fc0e4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:56.041657 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026b24c-702c-45e0-bc83-31547f56146d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:56.043100 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b24c-702c-45e0-bc83-31547f56146d service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:56.043390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b24c-702c-45e0-bc83-31547f56146d service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:56.043666 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b24c-702c-45e0-bc83-31547f56146d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:56.055027 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026b24c-702c-45e0-bc83-31547f56146d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:56.056545 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026b24c-702c-45e0-bc83-31547f56146d service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:56.067556 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1388/2820] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:56.643396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e51f70-865a-4464-a586-bd8a0f2c90c2 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:56.720448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e51f70-865a-4464-a586-bd8a0f2c90c2 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:56.725110 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:56.725744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:56.726862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:56.726862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] PATH_INFO: `/v3/users/3a21475ebb1d45239f449ad643e772da` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:56.727613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a21475ebb1d45239f449ad643e772da)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:56.729617 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:56.742497 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=100 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:56.956893 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a21475ebb1d45239f449ad643e772da'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:56.969267 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a21475ebb1d45239f449ad643e772da'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:56.969985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a21475ebb1d45239f449ad643e772da'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:56.970614 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a21475ebb1d45239f449ad643e772da'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:57.061360 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a21475ebb1d45239f449ad643e772da'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:57.076238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1433/2821] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:01:56 2022] DELETE /identity/v3/users/3a21475ebb1d45239f449ad643e772da => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:57.081169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f91226-0514-4545-a4a3-f550a1b1e774 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:57.081745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f91226-0514-4545-a4a3-f550a1b1e774 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:57.081745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f91226-0514-4545-a4a3-f550a1b1e774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:57.101977 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-f7f91226-0514-4545-a4a3-f550a1b1e774 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:57.103235 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f7f91226-0514-4545-a4a3-f550a1b1e774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:57.164513 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1389/2822] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:01:57.174234 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:57.175702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:57.236605 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1b524-e740-43f0-a26d-819c0d973689 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:57.240340 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00604373-67df-4962-8e7d-8a26ed922049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:57.240960 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604373-67df-4962-8e7d-8a26ed922049 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:57.241247 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604373-67df-4962-8e7d-8a26ed922049 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:57.241527 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604373-67df-4962-8e7d-8a26ed922049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:57.251182 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00604373-67df-4962-8e7d-8a26ed922049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:57.252657 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00604373-67df-4962-8e7d-8a26ed922049 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:57.264100 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1434/2823] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:57.731938 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f91226-0514-4545-a4a3-f550a1b1e774 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:57.735589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f7f91226-0514-4545-a4a3-f550a1b1e774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:57.743280 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:57.747410 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f91226-0514-4545-a4a3-f550a1b1e774 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:57.752256 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:57.753098 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:57.753390 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:57.753672 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] PATH_INFO: `/v3/projects/3f7fa945bb1d4fa9aa2038cb14f89936` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:57.755320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f7fa945bb1d4fa9aa2038cb14f89936)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:57.757274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:57.886360 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f7fa945bb1d4fa9aa2038cb14f89936', 'request_id': 'req-d386e76f-d550-4a29-a086-0ba8447fd870'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:57.891980 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f7fa945bb1d4fa9aa2038cb14f89936', 'request_id': 'req-d386e76f-d550-4a29-a086-0ba8447fd870'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:57.899543 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f7fa945bb1d4fa9aa2038cb14f89936', 'request_id': 'req-d386e76f-d550-4a29-a086-0ba8447fd870'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:58.027460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] The token cache is being invalidate because project 3f7fa945bb1d4fa9aa2038cb14f89936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:01:58.027690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:58.028303 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:58.029803 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1390/2824] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:01:57 2022] DELETE /identity/v3/projects/3f7fa945bb1d4fa9aa2038cb14f89936 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:58.033775 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00604373-67df-4962-8e7d-8a26ed922049 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:58.044363 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00604373-67df-4962-8e7d-8a26ed922049 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:58.103101 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd51219-f0bd-431c-90e2-4d7cb3e3554a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:58.103748 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd51219-f0bd-431c-90e2-4d7cb3e3554a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:58.104064 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd51219-f0bd-431c-90e2-4d7cb3e3554a admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:58.104349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd51219-f0bd-431c-90e2-4d7cb3e3554a admin admin] PATH_INFO: `/v3/users/52554d78e12843bca66d2dd10e0fa41e` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:58.105506 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd51219-f0bd-431c-90e2-4d7cb3e3554a admin admin] RBAC: Authorizing `identity:delete_user(user_id=52554d78e12843bca66d2dd10e0fa41e)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:58.107440 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd51219-f0bd-431c-90e2-4d7cb3e3554a admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:58.292244 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:58.304895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-abd51219-f0bd-431c-90e2-4d7cb3e3554a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52554d78e12843bca66d2dd10e0fa41e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:58.321815 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-abd51219-f0bd-431c-90e2-4d7cb3e3554a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52554d78e12843bca66d2dd10e0fa41e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:58.322629 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-abd51219-f0bd-431c-90e2-4d7cb3e3554a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52554d78e12843bca66d2dd10e0fa41e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:58.323237 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-abd51219-f0bd-431c-90e2-4d7cb3e3554a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52554d78e12843bca66d2dd10e0fa41e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:58.392548 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-abd51219-f0bd-431c-90e2-4d7cb3e3554a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52554d78e12843bca66d2dd10e0fa41e'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:58.406309 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1435/2825] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:01:58 2022] DELETE /identity/v3/users/52554d78e12843bca66d2dd10e0fa41e => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:58.845819 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:58.847431 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:58.864607 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386e76f-d550-4a29-a086-0ba8447fd870 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:58.922876 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:58.923512 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:58.923800 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:58.924107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] PATH_INFO: `/v3/projects/cda9e45c6bd749b680bbba30a9ef6b7f` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:58.925274 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda9e45c6bd749b680bbba30a9ef6b7f)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:58.927257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:59.050797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda9e45c6bd749b680bbba30a9ef6b7f', 'request_id': 'req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:59.057121 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda9e45c6bd749b680bbba30a9ef6b7f', 'request_id': 'req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:59.063886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda9e45c6bd749b680bbba30a9ef6b7f', 'request_id': 'req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:59.175763 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] The token cache is being invalidate because project cda9e45c6bd749b680bbba30a9ef6b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:01:59.176012 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:59.177197 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-2fc6c037-0da6-47fe-ad9d-afe819b6ef87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:59.178733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1391/2826] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:01:58 2022] DELETE /identity/v3/projects/cda9e45c6bd749b680bbba30a9ef6b7f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:59.183708 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f02ec4-86e7-4ed5-b064-bb0491039178 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:59.185323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f02ec4-86e7-4ed5-b064-bb0491039178 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:59.185323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f02ec4-86e7-4ed5-b064-bb0491039178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:59.204963 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-47f02ec4-86e7-4ed5-b064-bb0491039178 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:59.206195 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-47f02ec4-86e7-4ed5-b064-bb0491039178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:59.265447 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1436/2827] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:59.328872 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de21226-94cd-47cf-bbee-871d6a421071 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:59.329165 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de21226-94cd-47cf-bbee-871d6a421071 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:59.329452 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de21226-94cd-47cf-bbee-871d6a421071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:59.349409 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-3de21226-94cd-47cf-bbee-871d6a421071 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:59.350625 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3de21226-94cd-47cf-bbee-871d6a421071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:59.358602 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1392/2828] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:59.363029 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f02ec4-86e7-4ed5-b064-bb0491039178 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:59.373055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f02ec4-86e7-4ed5-b064-bb0491039178 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:59.377834 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0f42e-231d-4a68-99f0-8223b93b6cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:59.378453 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f42e-231d-4a68-99f0-8223b93b6cea admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:59.378742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f42e-231d-4a68-99f0-8223b93b6cea admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:59.379021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f42e-231d-4a68-99f0-8223b93b6cea admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:59.380225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0f42e-231d-4a68-99f0-8223b93b6cea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:01:59.380534 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0f42e-231d-4a68-99f0-8223b93b6cea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:59.382153 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0f42e-231d-4a68-99f0-8223b93b6cea admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:59.395011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1437/2829] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 16:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:01:59.400407 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de21226-94cd-47cf-bbee-871d6a421071 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:59.410882 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de21226-94cd-47cf-bbee-871d6a421071 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:59.416053 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb0b7-8535-4b72-ac63-66e4db7d8faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:59.416721 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb0b7-8535-4b72-ac63-66e4db7d8faa admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:59.417003 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb0b7-8535-4b72-ac63-66e4db7d8faa admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:59.417279 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb0b7-8535-4b72-ac63-66e4db7d8faa admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:59.417916 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fb0b7-8535-4b72-ac63-66e4db7d8faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:59.419894 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fb0b7-8535-4b72-ac63-66e4db7d8faa admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:59.430490 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:59.527025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1393/2830] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 16:01:59 2022] POST /identity/v3/projects => generated 359 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:59.531736 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0f42e-231d-4a68-99f0-8223b93b6cea admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:59.542984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0f42e-231d-4a68-99f0-8223b93b6cea admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:59.600653 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c82494-817c-4b8f-a1f3-7f389eb845ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:59.601268 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c82494-817c-4b8f-a1f3-7f389eb845ff admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:59.601587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c82494-817c-4b8f-a1f3-7f389eb845ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:59.601973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c82494-817c-4b8f-a1f3-7f389eb845ff admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:59.602613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c82494-817c-4b8f-a1f3-7f389eb845ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:59.604586 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c82494-817c-4b8f-a1f3-7f389eb845ff admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:59.713907 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1438/2831] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 16:01:59 2022] POST /identity/v3/users => generated 280 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:59.717887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb0b7-8535-4b72-ac63-66e4db7d8faa admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:59.729116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb0b7-8535-4b72-ac63-66e4db7d8faa admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:59.733959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e334b-e28d-40d5-bd50-71775b6aaac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:59.734571 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e334b-e28d-40d5-bd50-71775b6aaac8 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:59.734854 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e334b-e28d-40d5-bd50-71775b6aaac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:59.735130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e334b-e28d-40d5-bd50-71775b6aaac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:59.735780 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e334b-e28d-40d5-bd50-71775b6aaac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:59.737429 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e334b-e28d-40d5-bd50-71775b6aaac8 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:59.747754 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1394/2832] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:01:59.751805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c82494-817c-4b8f-a1f3-7f389eb845ff admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:59.762219 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c82494-817c-4b8f-a1f3-7f389eb845ff admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:59.767015 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fc8c3-1168-4675-be08-12b0bc94f76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:59.767616 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc8c3-1168-4675-be08-12b0bc94f76d admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:59.768552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc8c3-1168-4675-be08-12b0bc94f76d admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:59.768552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc8c3-1168-4675-be08-12b0bc94f76d admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:59.769134 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fc8c3-1168-4675-be08-12b0bc94f76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:59.771270 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fc8c3-1168-4675-be08-12b0bc94f76d admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:59.782415 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1439/2833] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:01:59.787241 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e334b-e28d-40d5-bd50-71775b6aaac8 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:59.797586 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e334b-e28d-40d5-bd50-71775b6aaac8 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:59.802743 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f304b22-1997-4fa6-83d5-91278c230602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:59.803531 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f304b22-1997-4fa6-83d5-91278c230602 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:59.803982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f304b22-1997-4fa6-83d5-91278c230602 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:59.804408 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f304b22-1997-4fa6-83d5-91278c230602 admin admin] PATH_INFO: `/v3/projects/c224399f209441078efc9a25711af927/users/13db7c9b5d6c4b52adcedf2f0598da63/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:59.820265 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f304b22-1997-4fa6-83d5-91278c230602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c224399f209441078efc9a25711af927, user_id=13db7c9b5d6c4b52adcedf2f0598da63, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:59.823247 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f304b22-1997-4fa6-83d5-91278c230602 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:59.942418 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1395/2834] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:01:59 2022] PUT /identity/v3/projects/c224399f209441078efc9a25711af927/users/13db7c9b5d6c4b52adcedf2f0598da63/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:01:59.952761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082a1e4-7016-4f53-a63a-35316c006624 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:59.953055 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082a1e4-7016-4f53-a63a-35316c006624 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:59.953330 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082a1e4-7016-4f53-a63a-35316c006624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:59.983014 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-c082a1e4-7016-4f53-a63a-35316c006624 None None] MFA Rules not processed for user `13db7c9b5d6c4b52adcedf2f0598da63`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:59.984209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c082a1e4-7016-4f53-a63a-35316c006624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:59.988193 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:00.049199 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1440/2835] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:01:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:02:00.086147 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f304b22-1997-4fa6-83d5-91278c230602 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:00.088065 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-8f304b22-1997-4fa6-83d5-91278c230602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:00.152094 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f304b22-1997-4fa6-83d5-91278c230602 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:00.155936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ec079-4b51-4ae4-a51b-9927c101eac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:00.156582 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ec079-4b51-4ae4-a51b-9927c101eac4 service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:00.156866 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ec079-4b51-4ae4-a51b-9927c101eac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:00.157146 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ec079-4b51-4ae4-a51b-9927c101eac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:00.167125 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ec079-4b51-4ae4-a51b-9927c101eac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:00.168641 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ec079-4b51-4ae4-a51b-9927c101eac4 service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:00.179713 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1396/2836] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:00.195396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082a1e4-7016-4f53-a63a-35316c006624 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:00.267089 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082a1e4-7016-4f53-a63a-35316c006624 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:00.272903 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:00.273565 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:00.274262 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:00.274583 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] PATH_INFO: `/v3/users/13db7c9b5d6c4b52adcedf2f0598da63` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:00.275786 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13db7c9b5d6c4b52adcedf2f0598da63)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:00.277837 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:00.509898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13db7c9b5d6c4b52adcedf2f0598da63'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:00.522984 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13db7c9b5d6c4b52adcedf2f0598da63'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:00.523633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13db7c9b5d6c4b52adcedf2f0598da63'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:00.524323 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13db7c9b5d6c4b52adcedf2f0598da63'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:00.626750 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13db7c9b5d6c4b52adcedf2f0598da63'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:00.640069 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1441/2837] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:02:00 2022] DELETE /identity/v3/users/13db7c9b5d6c4b52adcedf2f0598da63 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:02:00.645848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7aa11-9671-4a7a-bed0-e07e287dfa4d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:00.646149 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7aa11-9671-4a7a-bed0-e07e287dfa4d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:00.646425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7aa11-9671-4a7a-bed0-e07e287dfa4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:00.666746 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-3ef7aa11-9671-4a7a-bed0-e07e287dfa4d None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:02:00.668035 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3ef7aa11-9671-4a7a-bed0-e07e287dfa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:00.727484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1397/2838] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:02:00.737455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:00.738918 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:00.768752 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:00.800870 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4537c63-685a-4a0b-9cea-5553c5613372 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:00.804902 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36b8eb-d566-40ef-ac4f-ae843e40913c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:00.805511 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36b8eb-d566-40ef-ac4f-ae843e40913c service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:00.805788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36b8eb-d566-40ef-ac4f-ae843e40913c service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:00.806061 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36b8eb-d566-40ef-ac4f-ae843e40913c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:00.815449 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36b8eb-d566-40ef-ac4f-ae843e40913c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:00.817594 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36b8eb-d566-40ef-ac4f-ae843e40913c service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:00.827457 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1442/2839] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:01.289583 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7aa11-9671-4a7a-bed0-e07e287dfa4d None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:01.291351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3ef7aa11-9671-4a7a-bed0-e07e287dfa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:01.305056 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7aa11-9671-4a7a-bed0-e07e287dfa4d None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:01.310502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:01.311325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:01.311731 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:01.312321 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] PATH_INFO: `/v3/projects/c224399f209441078efc9a25711af927` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:01.314757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c224399f209441078efc9a25711af927)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:01.318326 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:01.421623 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c224399f209441078efc9a25711af927', 'request_id': 'req-87ac25ed-7b87-4735-b367-d2aebd9726fa'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:01.423621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:01.428187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c224399f209441078efc9a25711af927', 'request_id': 'req-87ac25ed-7b87-4735-b367-d2aebd9726fa'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:01.434496 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c224399f209441078efc9a25711af927', 'request_id': 'req-87ac25ed-7b87-4735-b367-d2aebd9726fa'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:01.528963 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] The token cache is being invalidate because project c224399f209441078efc9a25711af927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:02:01.529260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:01.529887 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-87ac25ed-7b87-4735-b367-d2aebd9726fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:01.531412 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1398/2840] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:02:01 2022] DELETE /identity/v3/projects/c224399f209441078efc9a25711af927 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:02:01.536967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19df5f-aa14-44ce-9fa5-514dc686009f None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:01.537480 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19df5f-aa14-44ce-9fa5-514dc686009f None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:01.537593 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19df5f-aa14-44ce-9fa5-514dc686009f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:01.557625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-2c19df5f-aa14-44ce-9fa5-514dc686009f None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:02:01.558825 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2c19df5f-aa14-44ce-9fa5-514dc686009f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:01.619307 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1443/2841] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:02:01.627113 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb84bdf-d465-46d2-bd5a-2a59b5606bac None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:01.627652 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb84bdf-d465-46d2-bd5a-2a59b5606bac None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:01.630785 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb84bdf-d465-46d2-bd5a-2a59b5606bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:01.649456 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-dbb84bdf-d465-46d2-bd5a-2a59b5606bac None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:02:01.650692 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-dbb84bdf-d465-46d2-bd5a-2a59b5606bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:01.659192 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1399/2842] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:02:01.664530 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19df5f-aa14-44ce-9fa5-514dc686009f None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:01.676901 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19df5f-aa14-44ce-9fa5-514dc686009f None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:01.683355 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cce227-7886-467c-a2ef-020ebca0ef28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:01.684284 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cce227-7886-467c-a2ef-020ebca0ef28 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:01.684706 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cce227-7886-467c-a2ef-020ebca0ef28 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:01.685117 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cce227-7886-467c-a2ef-020ebca0ef28 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:01.686105 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cce227-7886-467c-a2ef-020ebca0ef28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:02:01.686571 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cce227-7886-467c-a2ef-020ebca0ef28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:01.689242 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cce227-7886-467c-a2ef-020ebca0ef28 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:01.709357 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1444/2843] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 16:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:02:01.714357 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb84bdf-d465-46d2-bd5a-2a59b5606bac None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:01.726249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb84bdf-d465-46d2-bd5a-2a59b5606bac None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:01.733195 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544fe85-9d39-455b-a9ca-ecf37e379053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:01.734134 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544fe85-9d39-455b-a9ca-ecf37e379053 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:01.734572 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544fe85-9d39-455b-a9ca-ecf37e379053 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:01.735002 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544fe85-9d39-455b-a9ca-ecf37e379053 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:01.735905 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544fe85-9d39-455b-a9ca-ecf37e379053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:01.737892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544fe85-9d39-455b-a9ca-ecf37e379053 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:01.840348 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1400/2844] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 16:02:01 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:02:01.844638 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cce227-7886-467c-a2ef-020ebca0ef28 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:01.855727 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cce227-7886-467c-a2ef-020ebca0ef28 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:01.938185 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6e9c4-8c35-496b-b901-4f7bda1371e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:01.938813 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6e9c4-8c35-496b-b901-4f7bda1371e2 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:01.939098 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6e9c4-8c35-496b-b901-4f7bda1371e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:01.939378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6e9c4-8c35-496b-b901-4f7bda1371e2 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:01.940062 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6e9c4-8c35-496b-b901-4f7bda1371e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:01.942624 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6e9c4-8c35-496b-b901-4f7bda1371e2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:02.062208 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1445/2845] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 16:02:01 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:02:02.066701 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544fe85-9d39-455b-a9ca-ecf37e379053 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:02.078199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544fe85-9d39-455b-a9ca-ecf37e379053 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:02.082889 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbee955-b587-4f59-81a7-7b8b0180f665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:02.083498 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbee955-b587-4f59-81a7-7b8b0180f665 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:02.083783 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbee955-b587-4f59-81a7-7b8b0180f665 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:02.084215 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbee955-b587-4f59-81a7-7b8b0180f665 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:02.084964 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbee955-b587-4f59-81a7-7b8b0180f665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:02.086561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbee955-b587-4f59-81a7-7b8b0180f665 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:02.097857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1401/2846] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:02:02.102052 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6e9c4-8c35-496b-b901-4f7bda1371e2 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:02.112636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6e9c4-8c35-496b-b901-4f7bda1371e2 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:02.118136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10dbb5-4e73-4239-87a1-bf2a7cecf2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:02.118744 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10dbb5-4e73-4239-87a1-bf2a7cecf2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:02.119033 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10dbb5-4e73-4239-87a1-bf2a7cecf2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:02.119312 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10dbb5-4e73-4239-87a1-bf2a7cecf2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:02.119995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10dbb5-4e73-4239-87a1-bf2a7cecf2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:02.121716 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10dbb5-4e73-4239-87a1-bf2a7cecf2f2 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:02.134023 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1446/2847] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:02:02.138425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbee955-b587-4f59-81a7-7b8b0180f665 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:02.149857 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbee955-b587-4f59-81a7-7b8b0180f665 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:02.154865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4d886-e6a0-4949-955f-eb1c3688a511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:02.155688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4d886-e6a0-4949-955f-eb1c3688a511 admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:02.156084 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4d886-e6a0-4949-955f-eb1c3688a511 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:02.156518 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4d886-e6a0-4949-955f-eb1c3688a511 admin admin] PATH_INFO: `/v3/projects/d4aca222b23d447d84c26967902508ea/users/0df9431cefb742cc980351e7f8df8950/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:02.174514 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4d886-e6a0-4949-955f-eb1c3688a511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4aca222b23d447d84c26967902508ea, user_id=0df9431cefb742cc980351e7f8df8950, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:02.177404 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4d886-e6a0-4949-955f-eb1c3688a511 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:02.250523 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1402/2848] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:02:02 2022] PUT /identity/v3/projects/d4aca222b23d447d84c26967902508ea/users/0df9431cefb742cc980351e7f8df8950/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:02:02.256509 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53014e-078c-45dc-9d1a-13b8fc40fe56 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:02.256801 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53014e-078c-45dc-9d1a-13b8fc40fe56 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:02.257114 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53014e-078c-45dc-9d1a-13b8fc40fe56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:02.277990 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-bc53014e-078c-45dc-9d1a-13b8fc40fe56 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:02:02.279238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-bc53014e-078c-45dc-9d1a-13b8fc40fe56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:02.311498 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:02.344375 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1447/2849] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:02:02.354347 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4d886-e6a0-4949-955f-eb1c3688a511 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:02.356163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7bc4d886-e6a0-4949-955f-eb1c3688a511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:02.415859 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4d886-e6a0-4949-955f-eb1c3688a511 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:02.419464 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091a0c3-ab18-41c3-a860-624df484177d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:02.420107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091a0c3-ab18-41c3-a860-624df484177d service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:02.420399 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091a0c3-ab18-41c3-a860-624df484177d service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:02.420671 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091a0c3-ab18-41c3-a860-624df484177d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:02.430320 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091a0c3-ab18-41c3-a860-624df484177d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:02.431792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091a0c3-ab18-41c3-a860-624df484177d service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:02.441509 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1403/2850] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:02:03.841295 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafb446-d50e-47d5-b4a2-832e68b99222 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:03.841802 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafb446-d50e-47d5-b4a2-832e68b99222 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:03.842041 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafb446-d50e-47d5-b4a2-832e68b99222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:03.882763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ffafb446-d50e-47d5-b4a2-832e68b99222 None None] MFA Rules not processed for user `0df9431cefb742cc980351e7f8df8950`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:02:03.883959 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ffafb446-d50e-47d5-b4a2-832e68b99222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:03.954637 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1448/2851] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:02:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:02:03.965613 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091a0c3-ab18-41c3-a860-624df484177d service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:03.967020 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d091a0c3-ab18-41c3-a860-624df484177d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:04.026341 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091a0c3-ab18-41c3-a860-624df484177d service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:04.029971 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53676f55-bed8-450c-909c-c8250da1826a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:04.030572 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53676f55-bed8-450c-909c-c8250da1826a service cinder] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:04.030902 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53676f55-bed8-450c-909c-c8250da1826a service cinder] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:04.031177 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53676f55-bed8-450c-909c-c8250da1826a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:04.039630 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53676f55-bed8-450c-909c-c8250da1826a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:04.041106 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53676f55-bed8-450c-909c-c8250da1826a service cinder] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:04.050116 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1404/2852] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:02:05.898636 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251db4b-d6eb-4817-84c3-5ae03c9b367e None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:05.899320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251db4b-d6eb-4817-84c3-5ae03c9b367e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:05.899320 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251db4b-d6eb-4817-84c3-5ae03c9b367e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:05.921718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e251db4b-d6eb-4817-84c3-5ae03c9b367e None None] MFA Rules not processed for user `0df9431cefb742cc980351e7f8df8950`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:02:05.921718 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e251db4b-d6eb-4817-84c3-5ae03c9b367e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:05.929362 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1449/2853] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:02:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:02:06.203191 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53676f55-bed8-450c-909c-c8250da1826a service cinder] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:06.215290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53676f55-bed8-450c-909c-c8250da1826a service cinder] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:06.219092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fd850-7e2c-4a94-a264-17dbe0feeb19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:06.219759 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fd850-7e2c-4a94-a264-17dbe0feeb19 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:06.220077 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fd850-7e2c-4a94-a264-17dbe0feeb19 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:06.220380 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fd850-7e2c-4a94-a264-17dbe0feeb19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:06.222117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-c66fd850-7e2c-4a94-a264-17dbe0feeb19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:06.232638 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fd850-7e2c-4a94-a264-17dbe0feeb19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:06.239126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fd850-7e2c-4a94-a264-17dbe0feeb19 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:06.243251 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1405/2854] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:15.914179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=47 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:16.239090 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=91,keystone:DELETE=2,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:19.019753 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251db4b-d6eb-4817-84c3-5ae03c9b367e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:19.021232 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e251db4b-d6eb-4817-84c3-5ae03c9b367e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:19.094809 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251db4b-d6eb-4817-84c3-5ae03c9b367e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:19.100998 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbfdce-d0a9-458c-abe3-4d43c7f31500 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:19.101635 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfdce-d0a9-458c-abe3-4d43c7f31500 service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:19.101917 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfdce-d0a9-458c-abe3-4d43c7f31500 service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:19.102192 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfdce-d0a9-458c-abe3-4d43c7f31500 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:19.103952 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-aafbfdce-d0a9-458c-abe3-4d43c7f31500 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:19.116566 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbfdce-d0a9-458c-abe3-4d43c7f31500 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:19.118144 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbfdce-d0a9-458c-abe3-4d43c7f31500 service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:19.129302 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1450/2855] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:26.853765 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a2601-eae5-4cc5-8be5-16c29b1de8a9 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:26.854329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a2601-eae5-4cc5-8be5-16c29b1de8a9 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:26.854329 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a2601-eae5-4cc5-8be5-16c29b1de8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:26.875174 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-cd9a2601-eae5-4cc5-8be5-16c29b1de8a9 None None] MFA Rules not processed for user `0df9431cefb742cc980351e7f8df8950`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:02:26.876484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cd9a2601-eae5-4cc5-8be5-16c29b1de8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:26.884292 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1406/2856] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:02:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:02:29.124542 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=34 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:36.869713 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:50.671612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df689bf-cee9-4b07-ba49-e5764a5fcdfc None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:50.677431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df689bf-cee9-4b07-ba49-e5764a5fcdfc None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:50.677431 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df689bf-cee9-4b07-ba49-e5764a5fcdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:50.693238 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-9df689bf-cee9-4b07-ba49-e5764a5fcdfc None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:02:50.694461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-9df689bf-cee9-4b07-ba49-e5764a5fcdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:50.777733 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1451/2857] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Aug 26 16:02:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:02:50.928092 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a2601-eae5-4cc5-8be5-16c29b1de8a9 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:50.929523 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-cd9a2601-eae5-4cc5-8be5-16c29b1de8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:51.006656 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a2601-eae5-4cc5-8be5-16c29b1de8a9 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:51.011668 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec8585-a627-49b3-8356-2b93b57c288b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:51.012307 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec8585-a627-49b3-8356-2b93b57c288b service swift] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:51.012592 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec8585-a627-49b3-8356-2b93b57c288b service swift] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:51.012952 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec8585-a627-49b3-8356-2b93b57c288b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:51.022451 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec8585-a627-49b3-8356-2b93b57c288b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:51.024061 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec8585-a627-49b3-8356-2b93b57c288b service swift] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:51.033900 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1407/2858] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 16:02:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 16:03:00.769012 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=25 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:01.030636 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=32 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:09.487973 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4f4fc-5016-4405-9b9a-47b2a507e25e None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:09.488462 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4f4fc-5016-4405-9b9a-47b2a507e25e None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:09.489223 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4f4fc-5016-4405-9b9a-47b2a507e25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:09.508611 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-0fa4f4fc-5016-4405-9b9a-47b2a507e25e None None] MFA Rules not processed for user `b24a6d0c1dbe49a18b3b8b6732a938be`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:09.509850 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0fa4f4fc-5016-4405-9b9a-47b2a507e25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:09.517299 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1452/2859] 10.222.0.98 () {58 vars in 967 bytes} [Fri Aug 26 16:03:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:09.529425 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec8585-a627-49b3-8356-2b93b57c288b service swift] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:09.540704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec8585-a627-49b3-8356-2b93b57c288b service swift] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:09.545858 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0df4c-7aa7-4eae-ad97-51181e90de10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fae47d6b7a7472a9d94636bb126f802', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:09.546459 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0df4c-7aa7-4eae-ad97-51181e90de10 service swift] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:09.546745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0df4c-7aa7-4eae-ad97-51181e90de10 service swift] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:09.547026 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0df4c-7aa7-4eae-ad97-51181e90de10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:09.556379 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0df4c-7aa7-4eae-ad97-51181e90de10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:09.558035 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0df4c-7aa7-4eae-ad97-51181e90de10 service swift] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:09.566187 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1408/2860] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Aug 26 16:03:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:03:12.757490 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4f4fc-5016-4405-9b9a-47b2a507e25e None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:12.758934 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-0fa4f4fc-5016-4405-9b9a-47b2a507e25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:12.843589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4f4fc-5016-4405-9b9a-47b2a507e25e None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:12.848816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f04b60-9eac-4b35-a7c1-aac7cb0739b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:12.849472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04b60-9eac-4b35-a7c1-aac7cb0739b6 service placement] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:12.849760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04b60-9eac-4b35-a7c1-aac7cb0739b6 service placement] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:12.850044 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04b60-9eac-4b35-a7c1-aac7cb0739b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:12.851816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-16f04b60-9eac-4b35-a7c1-aac7cb0739b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:12.864422 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f04b60-9eac-4b35-a7c1-aac7cb0739b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:12.866011 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f04b60-9eac-4b35-a7c1-aac7cb0739b6 service placement] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:12.876349 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1453/2861] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:12.904909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0df4c-7aa7-4eae-ad97-51181e90de10 service swift] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:12.919124 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0df4c-7aa7-4eae-ad97-51181e90de10 service swift] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:12.921635 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0aac87-db24-4758-8be2-dfcc04fe6597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:12.922312 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0aac87-db24-4758-8be2-dfcc04fe6597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:12.922609 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0aac87-db24-4758-8be2-dfcc04fe6597 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:12.922874 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0aac87-db24-4758-8be2-dfcc04fe6597 admin admin] PATH_INFO: `/v3/users/0df9431cefb742cc980351e7f8df8950` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:12.924071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0aac87-db24-4758-8be2-dfcc04fe6597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0df9431cefb742cc980351e7f8df8950)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:12.926573 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0aac87-db24-4758-8be2-dfcc04fe6597 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:13.153479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9f0aac87-db24-4758-8be2-dfcc04fe6597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0df9431cefb742cc980351e7f8df8950'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:13.154126 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9f0aac87-db24-4758-8be2-dfcc04fe6597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0df9431cefb742cc980351e7f8df8950'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:13.166694 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9f0aac87-db24-4758-8be2-dfcc04fe6597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0df9431cefb742cc980351e7f8df8950'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:13.236747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9f0aac87-db24-4758-8be2-dfcc04fe6597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0df9431cefb742cc980351e7f8df8950'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:13.245578 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-9f0aac87-db24-4758-8be2-dfcc04fe6597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0df9431cefb742cc980351e7f8df8950'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:13.252047 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1409/2862] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:12 2022] DELETE /identity/v3/users/0df9431cefb742cc980351e7f8df8950 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:13.564194 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f04b60-9eac-4b35-a7c1-aac7cb0739b6 service placement] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:13.575421 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f04b60-9eac-4b35-a7c1-aac7cb0739b6 service placement] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:13.635597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:13.636260 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:13.636545 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:13.636822 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] PATH_INFO: `/v3/projects/d4aca222b23d447d84c26967902508ea` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:13.637954 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4aca222b23d447d84c26967902508ea)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:13.639857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:13.787171 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4aca222b23d447d84c26967902508ea', 'request_id': 'req-8939afc2-5e86-420a-a20c-c7c436853df5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:13.888239 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4aca222b23d447d84c26967902508ea', 'request_id': 'req-8939afc2-5e86-420a-a20c-c7c436853df5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:13.895145 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4aca222b23d447d84c26967902508ea', 'request_id': 'req-8939afc2-5e86-420a-a20c-c7c436853df5'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:13.906109 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] The token cache is being invalidate because project d4aca222b23d447d84c26967902508ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:13.906403 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:13.907045 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-8939afc2-5e86-420a-a20c-c7c436853df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:13.908587 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1454/2863] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:13 2022] DELETE /identity/v3/projects/d4aca222b23d447d84c26967902508ea => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:13.913478 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d93526-98ba-419b-91ca-c709d248b49a None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:13.913947 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d93526-98ba-419b-91ca-c709d248b49a None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:13.914163 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d93526-98ba-419b-91ca-c709d248b49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:13.933397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-d2d93526-98ba-419b-91ca-c709d248b49a None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:13.934621 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d2d93526-98ba-419b-91ca-c709d248b49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:13.994611 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1410/2864] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:14.002552 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b66f1-a362-4749-abdc-a06f778d74af None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:14.002843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b66f1-a362-4749-abdc-a06f778d74af None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:14.003123 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b66f1-a362-4749-abdc-a06f778d74af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:14.023995 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-e66b66f1-a362-4749-abdc-a06f778d74af None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:14.025510 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-e66b66f1-a362-4749-abdc-a06f778d74af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:14.033575 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1455/2865] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:14.037869 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d93526-98ba-419b-91ca-c709d248b49a None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:14.047488 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d93526-98ba-419b-91ca-c709d248b49a None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:14.052009 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6084a9d-1274-4459-aa04-45f97c9ad0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:14.052687 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6084a9d-1274-4459-aa04-45f97c9ad0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:14.052962 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6084a9d-1274-4459-aa04-45f97c9ad0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:14.053234 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6084a9d-1274-4459-aa04-45f97c9ad0c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:14.053881 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6084a9d-1274-4459-aa04-45f97c9ad0c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:03:14.054182 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6084a9d-1274-4459-aa04-45f97c9ad0c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:14.055764 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6084a9d-1274-4459-aa04-45f97c9ad0c9 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:14.067904 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1411/2866] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 16:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:03:14.072378 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b66f1-a362-4749-abdc-a06f778d74af None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:14.081898 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b66f1-a362-4749-abdc-a06f778d74af None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:14.087267 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891556a0-3b21-41d6-a52b-bac14fdad4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:14.087920 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891556a0-3b21-41d6-a52b-bac14fdad4ad admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:14.088209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891556a0-3b21-41d6-a52b-bac14fdad4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:14.088519 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891556a0-3b21-41d6-a52b-bac14fdad4ad admin admin] PATH_INFO: `/v3/projects` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:14.089149 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891556a0-3b21-41d6-a52b-bac14fdad4ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:14.091175 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891556a0-3b21-41d6-a52b-bac14fdad4ad admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:14.095745 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:14.199373 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1456/2867] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:14 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:14.203720 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6084a9d-1274-4459-aa04-45f97c9ad0c9 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:14.216662 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6084a9d-1274-4459-aa04-45f97c9ad0c9 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:14.268349 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:14.281797 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9b40c-8e45-47ed-8f51-9e7a452062b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:14.282405 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b40c-8e45-47ed-8f51-9e7a452062b8 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:14.282686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b40c-8e45-47ed-8f51-9e7a452062b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:14.282961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b40c-8e45-47ed-8f51-9e7a452062b8 admin admin] PATH_INFO: `/v3/users` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:14.283593 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9b40c-8e45-47ed-8f51-9e7a452062b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:14.285604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9b40c-8e45-47ed-8f51-9e7a452062b8 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:14.396831 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1412/2868] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:14 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:14.401058 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891556a0-3b21-41d6-a52b-bac14fdad4ad admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:14.411646 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891556a0-3b21-41d6-a52b-bac14fdad4ad admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:14.416385 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4ab9f-02b5-4150-b009-bdb8430c6014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:14.416992 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ab9f-02b5-4150-b009-bdb8430c6014 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:14.417279 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ab9f-02b5-4150-b009-bdb8430c6014 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:14.417558 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ab9f-02b5-4150-b009-bdb8430c6014 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:14.418212 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4ab9f-02b5-4150-b009-bdb8430c6014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:14.419819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4ab9f-02b5-4150-b009-bdb8430c6014 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:14.430504 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1457/2869] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:14.434484 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9b40c-8e45-47ed-8f51-9e7a452062b8 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:14.445975 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9b40c-8e45-47ed-8f51-9e7a452062b8 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:14.450722 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbf6ab-cefa-4a3c-b453-e7148fac4272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:14.451325 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbf6ab-cefa-4a3c-b453-e7148fac4272 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:14.451608 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbf6ab-cefa-4a3c-b453-e7148fac4272 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:14.451902 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbf6ab-cefa-4a3c-b453-e7148fac4272 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:14.452676 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbf6ab-cefa-4a3c-b453-e7148fac4272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:14.454254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbf6ab-cefa-4a3c-b453-e7148fac4272 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:14.464737 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1413/2870] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:14.468640 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4ab9f-02b5-4150-b009-bdb8430c6014 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:14.478820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4ab9f-02b5-4150-b009-bdb8430c6014 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:14.483741 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a0eb2-d1c3-47c3-b1d2-c74655b9f9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:14.484495 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a0eb2-d1c3-47c3-b1d2-c74655b9f9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:14.484779 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a0eb2-d1c3-47c3-b1d2-c74655b9f9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:14.485059 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a0eb2-d1c3-47c3-b1d2-c74655b9f9d7 admin admin] PATH_INFO: `/v3/projects/643de70a02164f5f96de62f050c55980/users/437265b71f1b47cd96f873c0c3b1ae30/roles/0166494127b14615900b2c121908b2ba` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:14.502600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a0eb2-d1c3-47c3-b1d2-c74655b9f9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643de70a02164f5f96de62f050c55980, user_id=437265b71f1b47cd96f873c0c3b1ae30, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:14.506073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a0eb2-d1c3-47c3-b1d2-c74655b9f9d7 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:14.597176 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1458/2871] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:14 2022] PUT /identity/v3/projects/643de70a02164f5f96de62f050c55980/users/437265b71f1b47cd96f873c0c3b1ae30/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:14.609792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925b01-48da-4ee2-b483-667286f2d48d None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:14.610088 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925b01-48da-4ee2-b483-667286f2d48d None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:14.610383 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925b01-48da-4ee2-b483-667286f2d48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:14.640688 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-b7925b01-48da-4ee2-b483-667286f2d48d None None] MFA Rules not processed for user `437265b71f1b47cd96f873c0c3b1ae30`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:14.641923 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b7925b01-48da-4ee2-b483-667286f2d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:14.711774 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1414/2872] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:03:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:14.723028 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a0eb2-d1c3-47c3-b1d2-c74655b9f9d7 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:14.724884 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-615a0eb2-d1c3-47c3-b1d2-c74655b9f9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:14.784301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a0eb2-d1c3-47c3-b1d2-c74655b9f9d7 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:14.787988 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358486ce-ed43-4bb9-8297-1ac9c764cc71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1ff8aff9e048c6972843e0b50c4d45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:14.788600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358486ce-ed43-4bb9-8297-1ac9c764cc71 service cinder] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:14.788878 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358486ce-ed43-4bb9-8297-1ac9c764cc71 service cinder] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:14.789151 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358486ce-ed43-4bb9-8297-1ac9c764cc71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:14.797814 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358486ce-ed43-4bb9-8297-1ac9c764cc71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:14.799265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358486ce-ed43-4bb9-8297-1ac9c764cc71 service cinder] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:14.808401 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1459/2873] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:24.708340 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:24.804246 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:25.089463 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7925b01-48da-4ee2-b483-667286f2d48d None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:25.090886 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-b7925b01-48da-4ee2-b483-667286f2d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:25.160841 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7925b01-48da-4ee2-b483-667286f2d48d None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:25.165902 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d025b5e-4719-474b-83cc-5b8df57e51d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:25.166520 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d025b5e-4719-474b-83cc-5b8df57e51d3 service placement] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:25.166805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d025b5e-4719-474b-83cc-5b8df57e51d3 service placement] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:25.167093 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d025b5e-4719-474b-83cc-5b8df57e51d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:25.168878 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-7d025b5e-4719-474b-83cc-5b8df57e51d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:25.180585 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d025b5e-4719-474b-83cc-5b8df57e51d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:25.182169 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d025b5e-4719-474b-83cc-5b8df57e51d3 service placement] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:25.192239 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1415/2874] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:35.187878 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=32 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:36.078919 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358486ce-ed43-4bb9-8297-1ac9c764cc71 service cinder] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:36.143027 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358486ce-ed43-4bb9-8297-1ac9c764cc71 service cinder] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:36.147777 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:36.148477 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:36.148789 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:36.149039 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] PATH_INFO: `/v3/users/437265b71f1b47cd96f873c0c3b1ae30` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:36.150158 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=437265b71f1b47cd96f873c0c3b1ae30)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:36.152209 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:36.366798 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437265b71f1b47cd96f873c0c3b1ae30'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:36.380501 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437265b71f1b47cd96f873c0c3b1ae30'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:36.381230 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437265b71f1b47cd96f873c0c3b1ae30'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:36.381855 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437265b71f1b47cd96f873c0c3b1ae30'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:36.527648 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437265b71f1b47cd96f873c0c3b1ae30'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:36.542742 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1460/2875] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:36 2022] DELETE /identity/v3/users/437265b71f1b47cd96f873c0c3b1ae30 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:36.547185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee7e90-d9f7-4a39-9633-7b868aa5dd19 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:36.547673 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee7e90-d9f7-4a39-9633-7b868aa5dd19 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:36.547792 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee7e90-d9f7-4a39-9633-7b868aa5dd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:36.567167 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-f5ee7e90-d9f7-4a39-9633-7b868aa5dd19 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:36.568416 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f5ee7e90-d9f7-4a39-9633-7b868aa5dd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:36.631547 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1416/2876] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:36.640021 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:36.641472 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:36.699572 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471b2eb-8dd8-4570-b29f-ac660317a7e0 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:36.703695 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459f122-f9a4-452b-83e6-c351cc3f40ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:36.704371 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459f122-f9a4-452b-83e6-c351cc3f40ae service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:36.704654 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459f122-f9a4-452b-83e6-c351cc3f40ae service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:36.704991 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459f122-f9a4-452b-83e6-c351cc3f40ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:36.715290 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459f122-f9a4-452b-83e6-c351cc3f40ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:36.717412 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459f122-f9a4-452b-83e6-c351cc3f40ae service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:36.726738 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1461/2877] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:37.163734 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee7e90-d9f7-4a39-9633-7b868aa5dd19 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:37.165249 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-f5ee7e90-d9f7-4a39-9633-7b868aa5dd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:37.177461 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee7e90-d9f7-4a39-9633-7b868aa5dd19 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:37.182273 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:37.186969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:37.186969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:37.186969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] PATH_INFO: `/v3/projects/643de70a02164f5f96de62f050c55980` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:37.186969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=643de70a02164f5f96de62f050c55980)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:37.186969 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:37.299117 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643de70a02164f5f96de62f050c55980', 'request_id': 'req-6db774fe-b801-49d9-b95b-687426f3a210'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.304951 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643de70a02164f5f96de62f050c55980', 'request_id': 'req-6db774fe-b801-49d9-b95b-687426f3a210'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.311210 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643de70a02164f5f96de62f050c55980', 'request_id': 'req-6db774fe-b801-49d9-b95b-687426f3a210'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.441597 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] The token cache is being invalidate because project 643de70a02164f5f96de62f050c55980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:37.441892 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.442525 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-6db774fe-b801-49d9-b95b-687426f3a210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.444788 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1417/2878] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:37 2022] DELETE /identity/v3/projects/643de70a02164f5f96de62f050c55980 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:37.450590 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d31c3-e688-4af2-9b06-dfd52fb5d5bb None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:37.450590 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d31c3-e688-4af2-9b06-dfd52fb5d5bb None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:37.450590 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d31c3-e688-4af2-9b06-dfd52fb5d5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:37.471600 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-828d31c3-e688-4af2-9b06-dfd52fb5d5bb None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:37.472857 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-828d31c3-e688-4af2-9b06-dfd52fb5d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:37.490265 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:37.531618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1462/2879] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:37.541262 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c76f94-475b-495b-a7b2-af2f85cc25c0 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:37.541551 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c76f94-475b-495b-a7b2-af2f85cc25c0 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:37.541961 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c76f94-475b-495b-a7b2-af2f85cc25c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:37.561775 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-77c76f94-475b-495b-a7b2-af2f85cc25c0 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:37.562983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-77c76f94-475b-495b-a7b2-af2f85cc25c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:37.571632 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1418/2880] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:37.575979 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d31c3-e688-4af2-9b06-dfd52fb5d5bb None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:37.585625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d31c3-e688-4af2-9b06-dfd52fb5d5bb None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:37.590256 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3cdce-d95a-4aec-8fbc-e53f472bedc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:37.590854 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3cdce-d95a-4aec-8fbc-e53f472bedc1 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:37.591137 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3cdce-d95a-4aec-8fbc-e53f472bedc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:37.591411 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3cdce-d95a-4aec-8fbc-e53f472bedc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:37.592084 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3cdce-d95a-4aec-8fbc-e53f472bedc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:03:37.592392 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3cdce-d95a-4aec-8fbc-e53f472bedc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:37.593980 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3cdce-d95a-4aec-8fbc-e53f472bedc1 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:37.605618 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1463/2881] 10.222.0.98 () {58 vars in 1153 bytes} [Fri Aug 26 16:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:03:37.615976 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c76f94-475b-495b-a7b2-af2f85cc25c0 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:37.625363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c76f94-475b-495b-a7b2-af2f85cc25c0 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:37.630983 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f0c53-ef34-4c25-9917-5fea38b57bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:37.631589 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f0c53-ef34-4c25-9917-5fea38b57bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:37.631891 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f0c53-ef34-4c25-9917-5fea38b57bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:37.632180 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f0c53-ef34-4c25-9917-5fea38b57bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:37.632810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f0c53-ef34-4c25-9917-5fea38b57bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:37.634812 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f0c53-ef34-4c25-9917-5fea38b57bc4 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:37.877495 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1419/2882] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:37 2022] POST /identity/v3/projects => generated 371 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:37.883354 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3cdce-d95a-4aec-8fbc-e53f472bedc1 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:37.893862 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3cdce-d95a-4aec-8fbc-e53f472bedc1 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:37.949609 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afb1ef-3ada-46df-9833-8ff66ba875d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:37.950205 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afb1ef-3ada-46df-9833-8ff66ba875d5 admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:37.950489 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afb1ef-3ada-46df-9833-8ff66ba875d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:37.950772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afb1ef-3ada-46df-9833-8ff66ba875d5 admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:37.951399 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98afb1ef-3ada-46df-9833-8ff66ba875d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:37.953369 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98afb1ef-3ada-46df-9833-8ff66ba875d5 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:38.042661 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1464/2883] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:37 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:38.046909 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f0c53-ef34-4c25-9917-5fea38b57bc4 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:38.057745 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f0c53-ef34-4c25-9917-5fea38b57bc4 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:38.062199 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fdacc-2093-4d99-bd81-0fc1452c7e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:38.062791 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fdacc-2093-4d99-bd81-0fc1452c7e38 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:38.063211 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fdacc-2093-4d99-bd81-0fc1452c7e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:38.063530 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fdacc-2093-4d99-bd81-0fc1452c7e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:38.064232 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fdacc-2093-4d99-bd81-0fc1452c7e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:38.065838 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fdacc-2093-4d99-bd81-0fc1452c7e38 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:38.075936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1420/2884] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:38.079705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afb1ef-3ada-46df-9833-8ff66ba875d5 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:38.089753 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afb1ef-3ada-46df-9833-8ff66ba875d5 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:38.094468 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb2afb-c979-4c87-95a6-36aa1309d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:38.095053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb2afb-c979-4c87-95a6-36aa1309d822 admin admin] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:38.095354 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb2afb-c979-4c87-95a6-36aa1309d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:38.095612 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb2afb-c979-4c87-95a6-36aa1309d822 admin admin] PATH_INFO: `/v3/roles` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:38.096301 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb2afb-c979-4c87-95a6-36aa1309d822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:38.098189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb2afb-c979-4c87-95a6-36aa1309d822 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:38.109359 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1465/2885] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:38.113562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fdacc-2093-4d99-bd81-0fc1452c7e38 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:38.124138 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fdacc-2093-4d99-bd81-0fc1452c7e38 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:38.128796 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109cc2b-16eb-45ea-a4a9-e940c643366f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:38.129441 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109cc2b-16eb-45ea-a4a9-e940c643366f admin admin] REQUEST_METHOD: `PUT` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:38.129724 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109cc2b-16eb-45ea-a4a9-e940c643366f admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:38.129999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109cc2b-16eb-45ea-a4a9-e940c643366f admin admin] PATH_INFO: `/v3/projects/b7eea71358e140c29fa3df652ce95a6b/users/4b66de0ca9674a989cac17ca80b6cbb7/roles/0166494127b14615900b2c121908b2ba` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:38.144842 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109cc2b-16eb-45ea-a4a9-e940c643366f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7eea71358e140c29fa3df652ce95a6b, user_id=4b66de0ca9674a989cac17ca80b6cbb7, role_id=0166494127b14615900b2c121908b2ba)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:38.147548 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109cc2b-16eb-45ea-a4a9-e940c643366f admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:38.241670 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1421/2886] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:38 2022] PUT /identity/v3/projects/b7eea71358e140c29fa3df652ce95a6b/users/4b66de0ca9674a989cac17ca80b6cbb7/roles/0166494127b14615900b2c121908b2ba => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:03:38.246623 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de042b41-1873-445b-9a21-1c1a706b1b05 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:38.246910 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de042b41-1873-445b-9a21-1c1a706b1b05 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:38.247186 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de042b41-1873-445b-9a21-1c1a706b1b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:38.265962 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-de042b41-1873-445b-9a21-1c1a706b1b05 None None] MFA Rules not processed for user `3fea9e7c5b754acaa0233c15976ac3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:38.267169 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-de042b41-1873-445b-9a21-1c1a706b1b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:38.326364 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1466/2887] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:38.336604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109cc2b-16eb-45ea-a4a9-e940c643366f admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:38.338302 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-2109cc2b-16eb-45ea-a4a9-e940c643366f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:38.360198 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:38.394334 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109cc2b-16eb-45ea-a4a9-e940c643366f admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:38.398713 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131eff6-692d-406e-b070-f46ec97e364a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:38.399324 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131eff6-692d-406e-b070-f46ec97e364a service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:38.399610 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131eff6-692d-406e-b070-f46ec97e364a service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:38.399915 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131eff6-692d-406e-b070-f46ec97e364a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:38.410154 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6131eff6-692d-406e-b070-f46ec97e364a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:38.412216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6131eff6-692d-406e-b070-f46ec97e364a service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:38.421702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1422/2888] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:43.933794 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe875d2-9cea-48f8-83a3-1aade95b1135 None None] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:43.942408 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe875d2-9cea-48f8-83a3-1aade95b1135 None None] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:43.942766 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe875d2-9cea-48f8-83a3-1aade95b1135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:43.976448 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.auth.core [None req-ebe875d2-9cea-48f8-83a3-1aade95b1135 None None] MFA Rules not processed for user `4b66de0ca9674a989cac17ca80b6cbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=82214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:43.977705 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-ebe875d2-9cea-48f8-83a3-1aade95b1135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:43.999592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:44.056697 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1467/2889] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:03:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:44.069747 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131eff6-692d-406e-b070-f46ec97e364a service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:44.087936 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131eff6-692d-406e-b070-f46ec97e364a service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:44.096805 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59feaf2a-a1e8-4cfe-826c-12cdd240baad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:44.098183 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59feaf2a-a1e8-4cfe-826c-12cdd240baad admin admin] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:44.098686 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59feaf2a-a1e8-4cfe-826c-12cdd240baad admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:44.099705 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59feaf2a-a1e8-4cfe-826c-12cdd240baad admin admin] PATH_INFO: `/v3/projects` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:44.100826 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59feaf2a-a1e8-4cfe-826c-12cdd240baad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:44.103959 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59feaf2a-a1e8-4cfe-826c-12cdd240baad admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:44.243778 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1423/2890] 10.222.0.98 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:44 2022] POST /identity/v3/projects => generated 371 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:44.247987 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe875d2-9cea-48f8-83a3-1aade95b1135 None None] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:44.259628 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe875d2-9cea-48f8-83a3-1aade95b1135 None None] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:44.326633 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79b8ca-a9f7-43ba-b08b-66e125cb579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:44.327251 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79b8ca-a9f7-43ba-b08b-66e125cb579b admin admin] REQUEST_METHOD: `POST` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:44.327537 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79b8ca-a9f7-43ba-b08b-66e125cb579b admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:44.327816 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79b8ca-a9f7-43ba-b08b-66e125cb579b admin admin] PATH_INFO: `/v3/users` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:44.328484 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79b8ca-a9f7-43ba-b08b-66e125cb579b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:44.330461 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79b8ca-a9f7-43ba-b08b-66e125cb579b admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:44.483006 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1468/2891] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:44 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:44.487175 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59feaf2a-a1e8-4cfe-826c-12cdd240baad admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:44.498216 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59feaf2a-a1e8-4cfe-826c-12cdd240baad admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:44.502865 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654600d5-c809-49e8-afda-12af39ae8291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:44.503471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654600d5-c809-49e8-afda-12af39ae8291 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:44.503757 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654600d5-c809-49e8-afda-12af39ae8291 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:44.504071 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654600d5-c809-49e8-afda-12af39ae8291 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:44.504752 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654600d5-c809-49e8-afda-12af39ae8291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:44.506378 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654600d5-c809-49e8-afda-12af39ae8291 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:44.517438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1424/2892] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:44.522204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79b8ca-a9f7-43ba-b08b-66e125cb579b admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:44.532471 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79b8ca-a9f7-43ba-b08b-66e125cb579b admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:44.537597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6fdaa-3cde-44d8-b596-3c541b94f1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:44.538241 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6fdaa-3cde-44d8-b596-3c541b94f1fb admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:44.538526 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6fdaa-3cde-44d8-b596-3c541b94f1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:44.538804 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6fdaa-3cde-44d8-b596-3c541b94f1fb admin admin] PATH_INFO: `/v3/projects/5d05e062e6894989b8f9fc1b0bf65585/users/bbb29f30d7e74a419ab87f2a292388f3/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:44.553675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6fdaa-3cde-44d8-b596-3c541b94f1fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d05e062e6894989b8f9fc1b0bf65585, user_id=bbb29f30d7e74a419ab87f2a292388f3, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:44.556445 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6fdaa-3cde-44d8-b596-3c541b94f1fb admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:44.637216 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1469/2893] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:44 2022] PUT /identity/v3/projects/5d05e062e6894989b8f9fc1b0bf65585/users/bbb29f30d7e74a419ab87f2a292388f3/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:44.641309 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654600d5-c809-49e8-afda-12af39ae8291 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:44.652479 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654600d5-c809-49e8-afda-12af39ae8291 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:44.714588 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b9796-1037-4a6d-878e-2a0961e96f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:44.715212 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b9796-1037-4a6d-878e-2a0961e96f88 admin admin] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:44.715501 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b9796-1037-4a6d-878e-2a0961e96f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:44.715782 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b9796-1037-4a6d-878e-2a0961e96f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:44.716519 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b9796-1037-4a6d-878e-2a0961e96f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:44.718103 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b9796-1037-4a6d-878e-2a0961e96f88 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:44.728557 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1425/2894] 10.222.0.98 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:44.732880 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6fdaa-3cde-44d8-b596-3c541b94f1fb admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:44.743625 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6fdaa-3cde-44d8-b596-3c541b94f1fb admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:44.748613 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb0d3b-d60b-47ac-8ab4-86fc2c163a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:44.749207 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb0d3b-d60b-47ac-8ab4-86fc2c163a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:44.749493 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb0d3b-d60b-47ac-8ab4-86fc2c163a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:44.749772 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb0d3b-d60b-47ac-8ab4-86fc2c163a75 admin admin] PATH_INFO: `/v3/projects/5d05e062e6894989b8f9fc1b0bf65585/users/bbb29f30d7e74a419ab87f2a292388f3/roles/6ba5a5add2174983be37258aad1ee404` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:44.751717 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb0d3b-d60b-47ac-8ab4-86fc2c163a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d05e062e6894989b8f9fc1b0bf65585, user_id=bbb29f30d7e74a419ab87f2a292388f3, role_id=6ba5a5add2174983be37258aad1ee404)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:44.754396 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb0d3b-d60b-47ac-8ab4-86fc2c163a75 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:44.765195 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1470/2895] 10.222.0.98 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:44 2022] PUT /identity/v3/projects/5d05e062e6894989b8f9fc1b0bf65585/users/bbb29f30d7e74a419ab87f2a292388f3/roles/6ba5a5add2174983be37258aad1ee404 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:50.861769 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e83c9b-5a62-4bf5-a576-897addde36d5 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:50.862715 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e83c9b-5a62-4bf5-a576-897addde36d5 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:50.862995 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e83c9b-5a62-4bf5-a576-897addde36d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:50.892645 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-d2e83c9b-5a62-4bf5-a576-897addde36d5 None None] MFA Rules not processed for user `bbb29f30d7e74a419ab87f2a292388f3`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:50.893882 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-d2e83c9b-5a62-4bf5-a576-897addde36d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:50.971227 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1426/2896] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:03:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:50.990728 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb0d3b-d60b-47ac-8ab4-86fc2c163a75 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:50.993917 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-d0eb0d3b-d60b-47ac-8ab4-86fc2c163a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:51.079420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb0d3b-d60b-47ac-8ab4-86fc2c163a75 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:51.086236 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92456b94-a243-4fbf-9e13-1cd602114c1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:51.086872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456b94-a243-4fbf-9e13-1cd602114c1b service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:51.087198 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456b94-a243-4fbf-9e13-1cd602114c1b service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:51.087446 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456b94-a243-4fbf-9e13-1cd602114c1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:51.093140 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:51.150760 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92456b94-a243-4fbf-9e13-1cd602114c1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:51.152420 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92456b94-a243-4fbf-9e13-1cd602114c1b service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:51.163852 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1471/2897] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:54.305690 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad6f18-a263-4697-8336-18bfe9a58263 None None] REQUEST_METHOD: `POST` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:54.306260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad6f18-a263-4697-8336-18bfe9a58263 None None] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:54.306451 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad6f18-a263-4697-8336-18bfe9a58263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:54.328904 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.auth.core [None req-edad6f18-a263-4697-8336-18bfe9a58263 None None] MFA Rules not processed for user `4b66de0ca9674a989cac17ca80b6cbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=82215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:54.330130 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-edad6f18-a263-4697-8336-18bfe9a58263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:54.337996 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1427/2898] 10.222.0.98 () {58 vars in 958 bytes} [Fri Aug 26 16:03:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:01.172597 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=24 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:04.323078 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:30.093344 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92456b94-a243-4fbf-9e13-1cd602114c1b service nova] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:30.148972 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92456b94-a243-4fbf-9e13-1cd602114c1b service nova] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:30.152576 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a40f47-a1e8-4028-b703-36499416af23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:30.153179 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a40f47-a1e8-4028-b703-36499416af23 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:30.153466 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a40f47-a1e8-4028-b703-36499416af23 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:30.153749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a40f47-a1e8-4028-b703-36499416af23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:30.162985 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a40f47-a1e8-4028-b703-36499416af23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:30.164455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a40f47-a1e8-4028-b703-36499416af23 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:30.175888 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1472/2899] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:32.389773 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad6f18-a263-4697-8336-18bfe9a58263 None None] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:32.403454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad6f18-a263-4697-8336-18bfe9a58263 None None] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:32.409575 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d6ffe-5edc-40e1-a669-f989aa3ec6dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:32.410712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d6ffe-5edc-40e1-a669-f989aa3ec6dd service nova] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:32.410712 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d6ffe-5edc-40e1-a669-f989aa3ec6dd service nova] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:32.411019 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d6ffe-5edc-40e1-a669-f989aa3ec6dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:32.413536 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-431d6ffe-5edc-40e1-a669-f989aa3ec6dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:32.426950 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d6ffe-5edc-40e1-a669-f989aa3ec6dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:32.428704 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d6ffe-5edc-40e1-a669-f989aa3ec6dd service nova] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:32.440702 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1428/2900] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:40.169747 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=26 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:42.435254 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=6 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:25.250276 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a40f47-a1e8-4028-b703-36499416af23 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:25.259120 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a40f47-a1e8-4028-b703-36499416af23 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:25.262895 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c13fb-20ab-4c05-8ee7-49c771f64ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:25.263487 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c13fb-20ab-4c05-8ee7-49c771f64ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:25.263774 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c13fb-20ab-4c05-8ee7-49c771f64ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:25.264085 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c13fb-20ab-4c05-8ee7-49c771f64ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:25.265805 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-996c13fb-20ab-4c05-8ee7-49c771f64ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:25.276076 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c13fb-20ab-4c05-8ee7-49c771f64ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:25.277500 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c13fb-20ab-4c05-8ee7-49c771f64ae0 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:25.286354 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1473/2901] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:27.864992 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d6ffe-5edc-40e1-a669-f989aa3ec6dd service nova] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:27.874524 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d6ffe-5edc-40e1-a669-f989aa3ec6dd service nova] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:27.878422 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01596d-5806-4bc9-9f1d-cdca75bb7f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:27.879034 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01596d-5806-4bc9-9f1d-cdca75bb7f65 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:27.879314 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01596d-5806-4bc9-9f1d-cdca75bb7f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:27.879590 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01596d-5806-4bc9-9f1d-cdca75bb7f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:27.881430 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-3b01596d-5806-4bc9-9f1d-cdca75bb7f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:27.895438 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01596d-5806-4bc9-9f1d-cdca75bb7f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:27.896987 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01596d-5806-4bc9-9f1d-cdca75bb7f65 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:27.910913 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1429/2902] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:06:29.092675 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c13fb-20ab-4c05-8ee7-49c771f64ae0 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:29.094087 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-996c13fb-20ab-4c05-8ee7-49c771f64ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:29.104843 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c13fb-20ab-4c05-8ee7-49c771f64ae0 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:29.108592 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbba69d-3a48-4053-bad8-d409db6b1649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:29.109189 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba69d-3a48-4053-bad8-d409db6b1649 service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:29.109476 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba69d-3a48-4053-bad8-d409db6b1649 service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:29.109758 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba69d-3a48-4053-bad8-d409db6b1649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:29.112053 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-6cbba69d-3a48-4053-bad8-d409db6b1649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:29.125659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbba69d-3a48-4053-bad8-d409db6b1649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:29.127702 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbba69d-3a48-4053-bad8-d409db6b1649 service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:29.138872 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1474/2903] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:37.903374 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=6 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:39.133788 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=12 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:42.701653 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01596d-5806-4bc9-9f1d-cdca75bb7f65 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:42.711076 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01596d-5806-4bc9-9f1d-cdca75bb7f65 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:42.714957 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394b125-1b39-4f05-945d-3a8ff1281629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:42.715640 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394b125-1b39-4f05-945d-3a8ff1281629 service neutron] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:42.715982 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394b125-1b39-4f05-945d-3a8ff1281629 service neutron] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:42.716263 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394b125-1b39-4f05-945d-3a8ff1281629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:42.718023 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5394b125-1b39-4f05-945d-3a8ff1281629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:42.728449 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394b125-1b39-4f05-945d-3a8ff1281629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:42.729920 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394b125-1b39-4f05-945d-3a8ff1281629 service neutron] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:42.738290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1430/2904] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:42.815628 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbba69d-3a48-4053-bad8-d409db6b1649 service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:42.825155 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbba69d-3a48-4053-bad8-d409db6b1649 service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:42.829073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bbed7-d943-4c24-8942-16bca1cd529d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c6cad18359143868276eb9d79a3fd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:42.830243 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bbed7-d943-4c24-8942-16bca1cd529d service neutron] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:42.830536 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bbed7-d943-4c24-8942-16bca1cd529d service neutron] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:42.830819 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bbed7-d943-4c24-8942-16bca1cd529d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:42.832784 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f34bbed7-d943-4c24-8942-16bca1cd529d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:42.847761 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bbed7-d943-4c24-8942-16bca1cd529d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:42.849174 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bbed7-d943-4c24-8942-16bca1cd529d service neutron] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:42.860147 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1475/2905] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:42.920749 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394b125-1b39-4f05-945d-3a8ff1281629 service neutron] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:42.922143 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-5394b125-1b39-4f05-945d-3a8ff1281629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:42.992351 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394b125-1b39-4f05-945d-3a8ff1281629 service neutron] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:42.997587 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3d400-49f9-4316-91aa-3d6d228694d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:42.998219 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d400-49f9-4316-91aa-3d6d228694d1 service placement] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:42.998502 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d400-49f9-4316-91aa-3d6d228694d1 service placement] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:42.998778 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d400-49f9-4316-91aa-3d6d228694d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:43.000562 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-1fd3d400-49f9-4316-91aa-3d6d228694d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:43.012860 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3d400-49f9-4316-91aa-3d6d228694d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:43.014455 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3d400-49f9-4316-91aa-3d6d228694d1 service placement] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:43.024454 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1431/2906] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:52.855443 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=6 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:53.020129 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=38 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:08:16.042313 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bbed7-d943-4c24-8942-16bca1cd529d service neutron] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:08:16.043820 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f34bbed7-d943-4c24-8942-16bca1cd529d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:08:16.117166 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bbed7-d943-4c24-8942-16bca1cd529d service neutron] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:08:16.122967 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854683-b8d7-4f5b-b5e3-334fdfddeded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b01c21c38949938141eefdc4939385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:08:16.129183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854683-b8d7-4f5b-b5e3-334fdfddeded service nova] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:08:16.129183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854683-b8d7-4f5b-b5e3-334fdfddeded service nova] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:08:16.129183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854683-b8d7-4f5b-b5e3-334fdfddeded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:08:16.129183 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c9854683-b8d7-4f5b-b5e3-334fdfddeded service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:08:16.139082 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9854683-b8d7-4f5b-b5e3-334fdfddeded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:08:16.141367 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9854683-b8d7-4f5b-b5e3-334fdfddeded service nova] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:08:16.152178 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1476/2907] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:08:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:08:22.321999 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3d400-49f9-4316-91aa-3d6d228694d1 service placement] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:08:22.384257 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3d400-49f9-4316-91aa-3d6d228694d1 service placement] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:08:22.389363 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def9630-05a6-4c3f-a33f-71df407bfdc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:08:22.389996 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def9630-05a6-4c3f-a33f-71df407bfdc1 service placement] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:08:22.390281 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def9630-05a6-4c3f-a33f-71df407bfdc1 service placement] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:08:22.390561 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def9630-05a6-4c3f-a33f-71df407bfdc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:08:22.392399 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-9def9630-05a6-4c3f-a33f-71df407bfdc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:08:22.416025 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def9630-05a6-4c3f-a33f-71df407bfdc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:08:22.417604 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def9630-05a6-4c3f-a33f-71df407bfdc1 service placement] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:08:22.428271 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1432/2908] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:08:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:08:26.138875 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854683-b8d7-4f5b-b5e3-334fdfddeded service nova] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:08:26.224621 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854683-b8d7-4f5b-b5e3-334fdfddeded service nova] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:08:26.230455 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:08:26.231088 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:08:26.231373 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:08:26.231659 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] PATH_INFO: `/v3/users/4b66de0ca9674a989cac17ca80b6cbb7` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:08:26.232834 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b66de0ca9674a989cac17ca80b6cbb7)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:08:26.235316 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:08:26.402407 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b66de0ca9674a989cac17ca80b6cbb7'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:26.414964 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b66de0ca9674a989cac17ca80b6cbb7'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:26.415649 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b66de0ca9674a989cac17ca80b6cbb7'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:26.416280 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b66de0ca9674a989cac17ca80b6cbb7'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:26.485225 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b66de0ca9674a989cac17ca80b6cbb7'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:26.490749 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:08:26.500065 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1477/2909] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:08:26 2022] DELETE /identity/v3/users/4b66de0ca9674a989cac17ca80b6cbb7 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:08:26.778637 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def9630-05a6-4c3f-a33f-71df407bfdc1 service placement] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:08:26.790460 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def9630-05a6-4c3f-a33f-71df407bfdc1 service placement] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:08:26.850890 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:08:26.851560 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:08:26.851848 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:08:26.852159 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] PATH_INFO: `/v3/projects/b7eea71358e140c29fa3df652ce95a6b` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:08:26.853382 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7eea71358e140c29fa3df652ce95a6b)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:08:26.855290 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:08:26.984065 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7eea71358e140c29fa3df652ce95a6b', 'request_id': 'req-4ecd0760-b674-4bb2-9134-1f511ba85d6b'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:26.989810 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7eea71358e140c29fa3df652ce95a6b', 'request_id': 'req-4ecd0760-b674-4bb2-9134-1f511ba85d6b'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:26.996185 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7eea71358e140c29fa3df652ce95a6b', 'request_id': 'req-4ecd0760-b674-4bb2-9134-1f511ba85d6b'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:27.138012 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] The token cache is being invalidate because project b7eea71358e140c29fa3df652ce95a6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:08:27.138240 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:27.138853 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:27.140427 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1433/2910] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:08:26 2022] DELETE /identity/v3/projects/b7eea71358e140c29fa3df652ce95a6b => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:08:27.144940 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:08:27.146352 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:08:27.159620 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c56c1-8122-4b68-bc99-dc71922a08c6 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:08:27.216782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:08:27.217444 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:08:27.217763 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:08:27.218054 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] PATH_INFO: `/v3/users/bbb29f30d7e74a419ab87f2a292388f3` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:08:27.219233 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb29f30d7e74a419ab87f2a292388f3)` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:08:27.221204 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:08:27.431240 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb29f30d7e74a419ab87f2a292388f3'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:27.444353 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb29f30d7e74a419ab87f2a292388f3'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:27.445073 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb29f30d7e74a419ab87f2a292388f3'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:27.445757 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb29f30d7e74a419ab87f2a292388f3'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:27.542136 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.notifications [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb29f30d7e74a419ab87f2a292388f3'} {{(pid=82214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:27.555427 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1478/2911] 10.222.0.98 () {58 vars in 1224 bytes} [Fri Aug 26 16:08:27 2022] DELETE /identity/v3/users/bbb29f30d7e74a419ab87f2a292388f3 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:08:27.831850 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:08:27.833394 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:08:27.848260 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd0760-b674-4bb2-9134-1f511ba85d6b admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:08:27.853451 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:08:27.905475 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fea9e7c5b754acaa0233c15976ac3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86d26d8b55fb4a2b877ae676d9b1d379', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:08:27.906107 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:08:27.906419 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:08:27.906700 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] PATH_INFO: `/v3/projects/5d05e062e6894989b8f9fc1b0bf65585` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:08:27.907817 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d05e062e6894989b8f9fc1b0bf65585)` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:08:27.909741 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:08:28.062698 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d05e062e6894989b8f9fc1b0bf65585', 'request_id': 'req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:28.068233 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d05e062e6894989b8f9fc1b0bf65585', 'request_id': 'req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:28.074932 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d05e062e6894989b8f9fc1b0bf65585', 'request_id': 'req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3'} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:28.153313 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] The token cache is being invalidate because project 5d05e062e6894989b8f9fc1b0bf65585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:08:28.153603 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:28.154242 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.notifications [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:08:28.156366 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1434/2912] 10.222.0.98 () {58 vars in 1233 bytes} [Fri Aug 26 16:08:27 2022] DELETE /identity/v3/projects/5d05e062e6894989b8f9fc1b0bf65585 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:08:33.565818 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] Authenticating user token {{(pid=82214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:08:33.567288 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:08:33.641589 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c474e1e-8273-4dca-886d-a8458e7126c0 admin admin] Validating token access rules against request {{(pid=82214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:08:33.653259 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:08:33.703658 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35156e1-1380-4e63-9b7e-a22f235e8642 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:08:33.704374 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35156e1-1380-4e63-9b7e-a22f235e8642 service placement] REQUEST_METHOD: `GET` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:08:33.711332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35156e1-1380-4e63-9b7e-a22f235e8642 service placement] SCRIPT_NAME: `/identity` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:08:33.711332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35156e1-1380-4e63-9b7e-a22f235e8642 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:08:33.711332 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.fernet_utils [None req-f35156e1-1380-4e63-9b7e-a22f235e8642 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:08:33.718782 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35156e1-1380-4e63-9b7e-a22f235e8642 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:08:33.720405 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35156e1-1380-4e63-9b7e-a22f235e8642 service placement] RBAC: Authorization granted {{(pid=82214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:08:33.730599 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: [pid: 82214|app: 0|req: 1479/2913] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:08:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:08:38.149844 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:08:43.850835 dv-cinder-831247-24-iscsi devstack@keystone.service[82214]: DEBUG dbcounter [-] [82214] Writing DB stats keystone:SELECT=25 {{(pid=82214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:13:39.568471 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] Authenticating user token {{(pid=82215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:13:39.570055 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.fernet_utils [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:13:39.586115 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2083b-11ae-452d-93f1-9fa8e46f34f3 admin admin] Validating token access rules against request {{(pid=82215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:13:39.591742 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297985d-eb75-44ae-8ff1-8e4b649e0aa0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b28f987c90df4c19a12bb1819dc65cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19b186f6269a4406b3fc499a582a6e66', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:13:39.592397 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297985d-eb75-44ae-8ff1-8e4b649e0aa0 service placement] REQUEST_METHOD: `GET` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:13:39.592683 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297985d-eb75-44ae-8ff1-8e4b649e0aa0 service placement] SCRIPT_NAME: `/identity` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:13:39.592965 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297985d-eb75-44ae-8ff1-8e4b649e0aa0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:13:39.603135 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297985d-eb75-44ae-8ff1-8e4b649e0aa0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:13:39.608733 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297985d-eb75-44ae-8ff1-8e4b649e0aa0 service placement] RBAC: Authorization granted {{(pid=82215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:13:39.616735 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: [pid: 82215|app: 0|req: 1435/2914] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Aug 26 16:13:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:13:49.612207 dv-cinder-831247-24-iscsi devstack@keystone.service[82215]: DEBUG dbcounter [-] [82215] Writing DB stats keystone:SELECT=6 {{(pid=82215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}